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FOREWORD 



The National Forum on Education Statistics (the Forum) is pleased to present Traveling 
Through Time: The Forum Guide to Longitudinal Data Systems. This document, Book Two 
of Four: Planning and Developing an LDS, is the second installment of this Fomm series 
of guides on longitudinal data systems (LDS). One goal of the Forum is to improve the 
quality of education data gathered for use by policymakers and program decisionmakers. 
An approach to furthering this goal has been to pool the collective experiences of Fomm 
members to produce “best practice” guides in areas of high interest to those who collect, 
maintain, and use data about elementary and secondary education. Developing LDSs 
is one of those high-interest areas. These systems hold promise for enhancing both the 
way education agencies use data to serve students and the way they do business, from the 
policy level to the school office and into the classroom. 

LDSs are increasingly becoming the state of the art in education data. These systems 
move us from relying on blunt, aggregate, snapshot student data; to detailed and timely, 
student-level data that reflect the student’s entire academic history. An LDS makes it 
possible to not only monitor the success of individual students, but also to identify trends 
in those students’ education records. Freeing educators from guesswork and lessening the 
burden of painstaking data analysis, these systems provide powerful and timely insight 
about students and allow educators to tailor instruction to better meet individual needs. 

An LDS can reveal with great clarity what effects our policies, programs, and decisions have 
on schools. These systems allow agencies to track students across institutions to facilitate 
appropriate course placement and to determine who has transferred and who has dropped 
out. Longitudinal data systems also offer a new level of sophistication at the business 
level that can streamline operations; improve data quality; and free up valuable resources 
previously allocated to inefficient data entry, maintenance, and reporting practices. 

For these and others reasons, states should continue to introduce, develop, and 
expand their LDSs. The Traveling Through Time: The Forum Guide to Longitudinal Data 
Systems series is intended to help state and local education agencies meet the many 
challenges involved in developing robust systems, populating them with quality data, and 
using this new information to improve the education system. The series will introduce 
important topics, offer best practices, and direct the reader to additional resources related 
to the LDS development process. In sum, it is intended to help agencies establish LDSs 
that will have lasting, far-reaching impact on the education system and on students’ lives. 
For a description of the entire guide series, see appendix A. 



Book Two of Four: Planning and Developing an LDS 

This second book in the guide series delves into the planning, implementation, and 
evaluation phases of a longitudinal data system (LDS) project. It guides readers through 
the process of engaging a wide variety of stakeholders to create a vision for an LDS, build 
support for the undertaking, develop the system, and gauge how well it is meeting intended 
goals. 

> The introduction explains the purpose of book two, as well as the format 
and conventions used throughout the series. 
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> Chapter 1 introduces the information life cycle, a helpful concept when 
developing any data system. 

> Chapter 2 discusses the systems development life cycle, another useful 
concept for developing a data system. 

> Chapter 3 discusses the people and processes that characterize effective 
project management of an LDS development effort. 

> Chapter 4 provides guidance on engaging stakeholders to define the 
organization’s vision for an LDS that will meet end users’ needs. 

> Chapter 5 discusses the critical relationship between the state education 
agency (SEA) and districts, and offers best practices on how to strengthen 
these ties. 

> Chapter 6 discusses the self-assessment phase of the effort, during which an 
agency should identify its current system and functionalities. 

> Chapter 7 presents the concept of “enterprise architecture,” a helpful 
framework for planning and evaluating the agency’s current and desired data 
systems. 

> Chapter 8 discusses the needs-assessment phase, during which an agency 
should define its desired system based on stakeholder requirements. 

> Chapter 9 addresses how to identify the data elements that need to be 
collected to meet the stakeholders’ information needs. 

> Chapter 10 covers the important issues of interoperability and portability. 

> Chapter 11 discusses strategies for gaining sustained support for the system. 

> Chapter 12 addresses the need for, and approaches to, promoting the LDS 
and gaining grassroots support. 

> Chapter 13 discusses the choice between building an LDS in-house and 
hiring a vendor, addressing some of the pros and cons of each approach. 

> Chapter 14 provides an introduction to effective request for proposals (RFP) 
writing. 

> Chapter 15 addresses system evaluation, during which an agency assesses 
the “success” of the system based on many criteria. Findings should inform 
iterative system refinement. 

The appendices include an overview of the four books in this series, references, 
additional resources, an overview of information technology audits, and other relevant 
Forum and NCES resources. 
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The National Forum on Education Statistics 

The work of the Forum is a key aspect of the National Cooperative Education Statistics 
System (the Cooperative System). The Cooperative System was established to produce 
and maintain, with the cooperation of the states, comparable and uniform educational 
information and data that are useful for policymaking at the federal, state, and local levels. 
To assist in meeting this goal, the National Center for Education Statistics (NCES), within 
the U.S. Department of Education, established the National Forum on Education Statistics 
(the Forum) to improve the collection, reporting, and use of elementary and secondary 
education statistics. The Fomm deals with issues in education data policy, sponsors 
innovations in data collection and reporting, and provides technical assistance to improve 
state and local data systems. 



Development of Forum Products 

Members of the Fomm establish task forces to develop best-practice guides in data- 
related areas of interest to federal, state, and local education agencies. They are assisted 
in this work by NCES, but the content comes from the collective experience of the state 
and school district task force members who review all products iteratively throughout 
the development process. Documents prepared, reviewed, and approved by task force 
members undergo a formal public review. This public review consists of focus groups with 
representatives of the product’s intended audience, review sessions at relevant regional 
or national conferences, or technical reviews by acknowledged experts in the field. In 
addition, all draft documents are posted on the Forum website prior to publication so 
that any interested individuals or organizations can provide feedback. After the task force 
oversees the integration of public review comments and reviews the document a final time, 
publications are subject to examination by members of the Fomm standing committee 
sponsoring the project. Finally, the entire Forum (approximately 120 members) reviews and 
formally votes to approve all documents prior to publication. NCES provides final review 
and approval prior to publication. 
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INTRODUCTION 



his book, Planning and Developing an LDS, is the second in a four-part 
series about longitudinal data systems (LDS). The first book, What is an 
LDS?, focused on the fundamental questions of what an LDS is (and what it is not), what 
steps should be taken to achieve a sound system, what components make up an ideal 
system, and why such a system is of value in education. The present installment discusses 
the early stages of LDS development, and will help state and local education agencies 
through the process of determining what they want to accomplish with their LDS and 
what they will need in order to achieve these goals. The organization’s vision for an LDS 
should be heavily informed by the needs of a broad range of stakeholders. Throughout 
the systems development life cycle, policymakers and system developers need to engage in 
self-assessment, identifying the system they have before figuring out what type of system 
they want. Policymakers’ requirements should be driven by the needs of the education 
community, the costs involved given the legacy system and staff, and the institutional 
support for the project. Planners should ensure project sustainability by creating interest 
and sustained buy-in, and by securing long-term funding. Procurement planning must be 
done, that is, lining up a vendor or building the staffing capacity to constmct the system. 
In addition, having the right developers may not be enough: an informed commitment to 
building, using, and maintaining the LDS must permeate the organization to ensure long- 
term success. And, throughout the life of the system, thorough evaluation must be done 
on a regular basis to ensure continued data quality and user satisfaction. 



Figure 1 lays out the major issues discussed in each of the four books in this series. 
For more information on the purpose, format, and intended audience groups of this series, 
see Book One of Four: What is an LDS? 





Developing 
a successful 
LDS is 80% 
planning and 
20% building 
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Figure 1 . Summary of the Forum Guide to LDS Series 



Book l:What Is an LDS? 



i Understanding what an LDS is (and is not) 
i Appreciating the organizational steps needed to 
institute and effectively use an LDS 
i Identifying the technical features and 
capabilities of an effective LDS and the 
additional features that can enhance the 
system's utility 

i Recognizing the benefits of an LDS 



Book II: Planning and 
Developing an LDS 

■ Engaging stakeholders 

■ Describing the current system 

■ Envisioning the desired system 

■ Defining needs, including data and functionality 

■ Gaining buy-in and funding 

■ Building relationships 

■ Writing an RFP 

■ Building or buying a system or components 

■ Transferring knowledge (e.g., from developers 
to staff) 

■ Defining and measuring success 

■ Refining the system 



Book III: Effectively 
Managing LDS Data 

■ Defining governance structure 

■ Defining roles and responsibilities 

■ Collaborating to improve data quality and 
streamline operations 

■ Managing changes to the system 

■ Training staff to ensure data quality 

■ Auditing/validating data at all levels 

■ Establishing/following data standards 

■ Securing data to protect privacy 

■ Providing users access to key data 



Book IV: Advanced 
LDS Usage 

■ Collecting, storing, and delivering key data 

■ Developing useful reports to fulfill common 
data requests and needs 

■ Developing user-friendly data tools to facilitate 
access and analysis 

■ Training users to utilize the technology 

■ Building awareness, understanding, and 
analytical capacity 
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PLANNING PREREQUISITES: 

What to Think About Before 

DEVELOPING AN LDS 



Parti 



uilding an LDS can be a daunting task. Analysts who compare a newly 
E proposed system to the current information system environment often 

say, “You cannot get there from here.” Certainly the technological, organizational, and 
professional gaps between the information system that currently “is,” (the “here”) and the 
ideal system we hope will “be” in the future (the “there”) can be very discouraging. 
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Once the current reality is understood and the “future ideal” is clearly defined, 
an education agency can undertake the formal process commonly referred to as a “gap 
analysis,” which is often performed by third party experts. The agency can then begin to 
move from the present into the future. Later in this book, the processes of self- and needs- 
assessment are discussed to help readers identify “here” and “there.” But first, Part I reviews 
some prerequisites to the planning process. Understanding two conceptual frameworks— the 
life cycle of information and the systems development life cycle— will help developers 
and other stakeholders throughout the creation and refinement of the system. Then, the 
practical matter of establishing a project management team, and a process for driving and 
organizing the effort, will be addressed. 
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Chapter 1 

The Information Life Cycle 



O 



ne of the first concepts system designers and developers need to absorb is 
that information has a life cycle that begins with its creation and continues 
until its destruction. Understanding the life cycle of information is vital to evaluating the 
information systems policymakers have, and to designing the systems they need. Although 
the life cycle process can be described in many ways, the following verbs are used here to 
enumerate its different stages: define, create, collect, store, protect, use, share, and retire. 
(Note: The framework depicted in the graphic below applies to the information life cycle 
only. This Forum guide is not organized around this framework.) 



Figure 2. The information life cycle 




■ Define: Identify and define the data elements to be collected. 

■ Create: Create data (descriptive data exist and other data are 
created by events). 

■ Collect: Collect the data in the least burdensome manner. 

■ Store: Store data in accessible formats for efficient access and use. 

■ Protect: Secure data from individual or technological intrusion. 
Protect the privacy of individuals. 



■ Use: Use data for compliance, analyses, and education 
improvement. 

■ Share: Provide the public with appropriate data. 

■ Retire: Archive permanent records with historical or legal 
value. Destroy electronic records with little or no value to the 
data owner. 
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Define 

Before collecting data, organizations identify and define the data elements they need 
or want in order to comply with requirements or inform decisionmaking and business 
processes. Definition of data is not a precursor to the cycle, nor is it a one-time process. 



The Forum has more detailed information... 

... about the identification and definition of data elements: 




• NCES Handbooks Online: http://nces. ed.gov/programsAiandbook/index. asp 

• National Education Data Model: http://nces.ed.gov/forum/datamodel 



This stage should occur iteratively with every cycle to refine the data and their relationships 
so they better meet evolving needs. Only data that will be used should be defined (see 
chapter 9). 

Create and collect 

While some information already exists and will remain relatively constant over time (for 
example, student’s name, birth date, race, sex), other data are the products of events or 
activities (tests, course enrollments, discipline incidents, health events, etc.). Existing data 
will need to be added or crosswalked to the LDS, and new data will need to be collected 
as students progress through the education system. Thoroughly understanding the nature 
of the various types of data to be entered into the system and the collection processes 
that will be used is essential for recognizing the quality— completeness, timeliness, and 
accuracy— of the data, and thereby determining whether or not the data will be usable. It is 
also important to consider the burden and costs of acquiring and entering the data and the 
skills required to do this critical work (see chapter 9). 

Store and protect 

Determine the storage requirements for the data and the levels of protection they will 
require. The risk of exposure will vary based on the contents of the records. Risk has 
two components: the amount of harm that will be done if the data are accessed by an 
unauthorized party and the likelihood that this might happen. If the content of the records 
is such that little or no harm will be done if the system is breached, then the risk can be 
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considered low even if the likelihood of unauthorized use is high. But as the potential for 
harm increases, systems must provide higher levels of protection protocols to reduce the 
likelihood of unauthorized access to the data. 

Use 

Systems are built to facilitate the use of their data to improve the organization’s work 
and the students’ educational outcomes. The users of the data in the system should 
be the system builder’s primary “customers.” How the data are to be used, presented, 
and refreshed are just a few of the considerations that call for extensive requirements 
discussions. (Note: The purposes for which the data will be used should also be a central 
consideration in the “define” stage of the cycle.) 

Share 

How, when, under what circumstances, and with whom (individuals, organizations, other 
systems) the data will be shared is another set of questions to be deliberated. Sharing 
data often has legal and policy implications, such as freedom of information and privacy 
requirements. Clearly articulating all of these requirements, and the appropriate business 
mles to be followed, is necessary for legal and ethical compliance. 

Retire 

One of the last decisions in the life cycle of information comes when specific data cease to 
be accessed and used for the purposes for which they were originally collected and stored. 
When they are dormant yet still occupying valuable storage space, a decision must be 
made whether to archive or destroy the data. Some data by their nature are “eternal” and 
need to be properly and securely archived in case they are ever needed again (for example, 
transcript and financial data). Other records will eventually lose all of their value and 
should be destroyed in a manner consistent with their sensitivity. 

Before the design and development experts for any information system begin their 
work, system owners and planners should thoughtfully review all aspects of the life cycle 
of the information they propose to collect, store, and use. A thorough understanding 
of current information handling processes will provoke insights and suggestions for 
improvements and uncover difficulties that might otherwise not be discovered in a timely 
manner. Ultimately, the value and utility of the data will be greatly compromised if 
policymakers, system planners, and developers do not understand the whole life cycle and 
fail to ensure that the information is protected, collected, stored, and used correctly. 
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Chapter 2 



The Systems development 

Life Cycle 



D 



Developing anything more than a simple data retrieval system is a multiyear 
project. The number of system users, their requirements, the network of 
relationships, the complexities of ever-changing technology, and the personal politics 
involved with any system increases the importance of sound project management and an 
in-depth understanding of the life cycle of a systems development project. While an LDS 
is not just a project— it requires ongoing maintenance, use, training, etc.— thinking of LDS 
development as a project in its early phases can be useful. 



The systems development life cycle is "a project management 
technique that divides complex projects into smaller, more easily 
managed segments or phases." 

Source: Federal Financial Institutions Examination Council. 

Development and Acquisition. Available at 
http://www.ffiec.gov/ffiecinfobase/booklets/d_a/ 08 .html. 

Many words can be used to name the stages and substages in an information systems 
development effort, but for the purposes of this guide, the following verbs will be used to 
describe the whole life cycle: plan, analyze, design, develop, test, deploy, maintain, and 
evaluate. (Note: The framework depicted in the graphic on the following page applies to 
the systems development life cycle only. This Forum guide is not organized around this 
framework.) 

Plan 

The importance of thorough planning cannot be overemphasized. And neither can 
the need to take as much time as necessary to implement the plan successfully. For the 
purposes of this guide, “planning” includes working with a broad range of stakeholders 
who may be affected— or believe they may be affected— by the LDS in order to articulate 
the goals of the new system (see chapter 4), as well as assigning roles and responsibilities 
for project management (see chapter 3). Developing a communication plan that includes 
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Evaluate Maintain 



Figure 3. The Systems Development Life Cycle 
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■ Plan: 

• Articulate system goals and establish a high level view of the project. 

• Define responsibilities and roles. 

• Define project deliverables, budget, and schedule. 

■ Analyze: 

• Refine project goals and translate them into functional activities. 

• Analyze system users' information needs. 

• Determine project risks and risk-mitigation strategies. 

■ Design: 

• Describe desired features including screen layouts, business rules, 
process diagrams, system documentation, and collection methods. 

■ Develop: 

• Establish coding standards and naming conventions. 

• Write code to automate processes. 

• Create documentation. 

■ Test: 

• Develop testing plan, testing requirements, and testing schedule. 

• Run user acceptance tests and obtain feedback. 

• Correct defects and complete documentation manuals. 

■ Deploy: 

• Develop schedule and install applications into the production 
environment. 

• Run the new system parallel to the old one. 

• Verify data quality. 

• Establish a hard cutover date (a day of transition from old to new system). 

■ Maintain: 

• Develop an update schedule for hardware and software, as needed. 

• Maintain documentation. 

• Perform quality assurance and system security audits. 

■ Evaluate: 

• Monitor implementation and measure benefits. 

• Assess system's ability to meet users' initially identified and evolving 
information needs. 

• Identify necessary changes and return to planning stage of life cycle to 
implement them. 



all individuals and groups that may be affected by the education information system is 
essential. System development efforts might be seriously affected if a stakeholder becomes 
an active opponent of the effort. These potential surprises and other risks should be 
articulated and evaluated during the planning process. Data quality, sharing, and security 
strategies should be discussed— as well as the stakeholders and risks associated with these 
strategies. General goals should be defined by project deliverables with a set schedule and 
budget for each separate component as well as for the whole. At this point, the scope of 
the project— what is and is not to be included— should be clearly described and a method 
for managing changes established to monitor and control scope creep. In the planning 
stage, the current environment should be analyzed (see chapters 4, 6, and 7) and a clear 
picture of the future information system should emerge (see chapters 7-8). Anything 
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missed in the planning stage may go unnoticed until it is too late to fix or, at best, a return 
loop to the planning stage is required. Every project returns to the planning stage in the 
iterative process of system development; while this is unavoidable, the number of times 
repeating this stage is required can be reduced through thorough planning and by using the 
systems development life cycle the first time. 

Analyze 

Analyzing the information needs of all system users is a long and tedious process. It 
requires great patience and repetition to generate the input necessary to turn vague ideas 
of what would be useful information into definable and actionable system requirements. 
This is where the project goals and deliverables become defined in functional process 
terms. This is also where the project is critically examined to articulate and define the risks 
associated with the work to be done and what resources are needed. Each of these risks 
should be classified as high, medium, or low; and assigned a strategy for mitigation in case 
it materializes. Consideration should also be given to how data quality and security will 
be ensured (see chapters 4, 8, and 9). It may be useful to consider existing working models 
(such as successful in-house-developed and “off-the-shelf’ systems), while focusing on the 
most important ways the information in your desired system will be used and reported. 
Sharpening your understanding of the desired “there” system will make your analysis much 
more effective and shorten this stage of the process considerably. 



The Forum has more detailed information... 

... about these issues 

• Forum Unified Education Technology Suite (2005) 
http://nces.ed.gov/forum/pub_tech_suite.asp 

• Forum Guide to Decision Support Systems: A Resource for Educators (2006) 
http://nces. ed. gov/forum/pub_2006807. asp 

• Technology @ Your Fingertips (200 7 ) 
http://nces.ed.gov/pubs98/tech/index.asp 
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Design 

In the design stage, all the mental pictures created in the earlier stages are put into clear 
and completely documented forms. Business mles are articulated and refined, screen 
layouts are developed and improved, process diagrams are drawn and redrawn, and system 
documentation is carefully and completely kept. As with the previous two stages, the 
design stage often exposes issues or problems not previously discussed. This requires a 
return to the earlier stages in order to plan and analyze the newly exposed concerns. 

Develop 

When the design documents are complete, they are given to the developers to write the 
code that will automate the processes and produce the desired result. Coding standards 
and naming conventions are but two of the many considerations that must be made by the 
system developers. 

Test 

During the development stage, the testing stage begins and testing plans, user acceptance 
tests, and a testing schedule are developed. These tests are mn, shortcomings are 
discovered, and the system is modified to eliminate any defects. Any time or effort avoided 
at this stage may come back multiplied many times if a system defect survives to plague the 
system’s end users. 

Deploy 

In the deployment stage, the new system— with its hardware, software, and applications— is 
installed and often mn parallel to the old system until the acceptance criteria established 
earlier (in the plan and analysis phases) are met. System security is deployed and tested in 
real time situations, and data quality is checked and verified. 

Maintain 

When the system has met all required tests, it enters the maintenance stage. The system 
operations team performs periodic quality assurance and system security audits, updates 
hardware and software as needed, and maintains documentation. The maintenance 
stage continues as long as the system is operating and serves not only to keep the system 
mnning, but also to make incremental improvements. 

Evaluate 

The data system should be evaluated throughout its life cycle to ensure it is being 
implemented successfully, working optimally and cost effectively, providing measurable 
benefits, and meeting user needs as initially identified in the planning stages and as they 
evolve over time. Newly identified requirements will often warrant system changes beyond 
the level of “incremental,” requiring serious monitoring and evaluation that will produce 
system change requirements. Such changes should be routed through the entire systems 
development life cycle process. When it is determined that the existing system is no longer 
as effective or efficient as it should be, the system owners will go through the systems 
development life cycle again, beginning with planning for improvements to the existing 
system or the development of a new system. 
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project Management for an 

LDS IMPLEMENTATION 




^ reject management is a finite process, in which an organization launches, 
oversees, and concludes a project. As the management and enhancement 
of an established LDS is an ongoing endeavor that will support the education agency and 
external stakeholders, these ongoing stages should not be viewed as long-term projects. 
However, approaching the initial planning and development phases of the effort is 
appropriate. In these phases, the system is planned along a timeline, roles and deliverables 
are identified, resources are allotted and managed, communication is coordinated, and 
progress is overseen. Project management is especially important in LDS development 
because the effort touches numerous program areas within the organization, and 
perhaps many outside contractors as well. Therefore, it is vital to define the roles and 
responsibilities of the various stakeholders, their relationships, and timelines for each 
task. Effective project management will help the agency maintain a clear scope, manage 
expectations, and deliver components and functionalities successfully and on time. A clear, 
organized process is critical in such a complex project and will enhance the resulting LDS 
in terms of operations and ultimate utility. (Tennessee Department of Education) 



The Team 

Early on, a project management team should be established: a core group of education 
agency staff (and perhaps external participants) who will plan and manage the LDS effort. 
Though the makeup of this team will vary by agency and project, ideally its structure 
should be kept simple and include key leaders selected for their expertise and authority. 

> Executive sponsor 

A high-level authority within the organization (e.g., commissioner, assistant commissioner, 
or deputy commissioner) gives the project traction and deals with political issues that may 
impede or even doom the project. Specific responsibilities may include approving overall 
project scope and budget, securing funding for project costs, and overseeing project phases. 
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> Project manager 

The project manager (PM) drives the project management process. This leader should 
develop the project scope statement; define project deliverables, milestones, budget, and 
detailed work schedule; and assign project roles and responsibilities. In addition, the 
PM should establish the project management team, work with the sponsor to review the 
project plans, drive and lead team meetings, ensure that team members understand and are 
fulfilling their roles and responsibilities, and ensure that the project is ultimately completed 
in accordance with agency and external stakeholder expectations. While the agency’s 
chief information officer often fills this role, another agency leader may be selected as PM 
instead. (Tennessee Department of Education) 

> Data governance coordinator 

The leader of the data governance process works with the team to ensure the project is 
driven by education and programmatic needs rather than by IT. The coordinator’s primary 
responsibility should be to ensure the appropriate data managers are involved in the 
development and implementation phases as they are needed, for example, when the data 
they manage are rolled into the data warehouse or made available to users. 

> Program area data manager 

This standing seat on the team should be filled by different data managers as needed. 

As various data are incorporated into the system or involved in project activities, the 
appropriate manager should be engaged to ensure, for instance, that programmatic needs 
are reflected in the data model, definitions, and reports created from the LDS. 

> Core information technology staff 

In-house IT leaders either implement the system or, perhaps, work with vendors and 
sustain the system once outside developers have finished their work. 

> Vendor project manager 

If the agency is buying an LDS (in part or whole), the vendor project manager should keep 
the project on track and the agency up to speed on progress. 

> Database administrator or key technology lead 

If the LDS will be populated from existing data sources within the agency, a lead database 
administrator (DBA) or key technology lead should participate on the project management 
team as needed. The DBA understands how the source systems work, including their 
limitations and problems, and can help ensure effective incorporation of disparate data 
into the LDS. 



The Process 

The project management team should meet frequently, perhaps on a weekly basis, for 
planning, status updates, and discussion of problems and concerns. The following outlines 
some of the team’s key activities. 

> Planning the project scope 

In this first phase of project management, education goals should drive design— define 
your enterprise’s questions and requirements before developers build solutions. Engage 
stakeholders to define your current system as well as your desired system in terms of 
architecture, content, access, and use. Consider the work that will be necessary to ensure 
the quality of the data contained in the system, as well as the steps necessary to ensure 
data security and the privacy and confidentiality of sensitive student and staff records. 
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Devise a plan for evaluating the system’s success throughout the process, including the 
metrics that will be used to measure this success. Additionally the team should, based on 
broad stakeholder input, clarify how the system will ultimately be used, the training and 
professional development that will be needed, and the types of reports and technology 
solutions that will most help end users. 

> Identifying and prioritizing specific goals and deliverables 

Determine what needs to be done, and in what order, for each phase and portion of the 
development process. 

> Assigning work and roles 

Assign roles and responsibilities early on. This includes both in-house staff and external 
vendors, who should be assigned very clear roles and deliverables. 

> Monitoring work and deliverables 

Supervise the project work closely to ensure that activities are on track and to clear 
unexpected hurdles along the way. 

> Identifying critical issues 

Team members should communicate major problems that might affect implementation, 
and work together to find solutions as appropriate. Project management meetings should 
have a standing agenda item to identify these issues, determine a plan of action to address 
them, and update the team on progress towards the resolution of previously identified 
concerns. 

> Managing communication 

Keep stakeholders informed and involved in the LDS development and implementation 
effort in order to build and maintain interest. This will also help the team collect adequate 
input to create a system that meets user needs. Communication should not be limited to 
the project management team; rather, it should include coordinating updates about the 
project to the entire agency and all relevant stakeholders (legislators, state board, school 
districts, etc.). 
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Part II 



KNOWING WHAT YOU HAVE, 

Planning What you want 



fee 

| efore an education agency begins developing a new system— writing code or 

I an RFP— time should be taken for reflection or “discovery.” That is, before 
beginning its LDS journey, an agency should figure out where “here” is (what it has) and 
then carefully plan where “there” will be (what it will have). This often overlooked phase 
should begin at the start of the LDS development process with the engagement of a broad 
range of stakeholders, a thorough and collective self-assessment, and the identification of 
what the current system and environment look like. The resulting information, of course, 
will vary widely among organizations in terms of technology, applications, data, politics, 
resources, and other relevant factors. Once an agency has clearly established its “here,” it 
can pinpoint where it wants to go by thoroughly assessing stakeholders’ needs, then move 
on to carefully planning their desired system. When the self- and needs-assessments are 
complete, agencies should compare their “here” and “there” analyses to figure out how the 
current data system and organizational culture will need to change in order to fulfill the 
stakeholders’ stated needs. 

The following chapters address these cmcial early stages of the LDS development 
process. While the activities of engaging stakeholders, conducting self-assessments, and 
identifying needs are commonly referred to collectively as “needs assessment,” they are 
disaggregated here and discussed as distinct yet interrelated parts of the planning process. 




The early stages of "discovery" should not be rushed, so 
please drive slowly across the "Here to There Bridge." States 
that have done this agree that a year allows ample time 
for thorough self- and needs-assessment — to solicit broad 
stakeholder input and to carefully plan a new system. 
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Chapter 4 



Engaging Stakeholders: 
Bringing Everyone Along 



fsE 

I arly on, an education agency should pull together a broad range of 
stakeholders in a collaborative effort to define the organization’s LDS 
vision. Because a wide array of groups have a stake in the system, this early stage is critical 
for two primary reasons: it increases the usefulness and relevance of the system to users; 
and it increases the visibility and use of, and demand for, the system across the education 
community. 

Without diverse input from a range of perspectives, the resulting system may not be 
useful to all who should benefit from it. And if the system is not relevant to certain groups, 
it will not be used. Engaging users early in the design process increases the likelihood they 
will value and utilize the resulting system and, since they were given the opportunity in 
the design stages to ask for certain information, it will hopefully be more relevant to their 
efforts to improve student outcomes. Involving stakeholders in the LDS design process also 
serves as the first step in marketing the system (see chapter 12). This process educates users 
and gets them thinking about the system’s potential while spreading excitement, increasing 
buy-in, and fostering lasting executive and grassroots support for the project. 



Mix it up 

Enlist stakeholders who vary in terms of: 

• interest in the project; 

• expertise (e.g., business and technology specialists); 

• responsibilities (e.g., data collectors and data providers); 

• level of government (e.g., state, regional, or district); 

• geography; 

• benefits to be gained; and 

• perspective (e.g., insiders and outsiders). 
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See the forest and the trees 



In addition to talking about data assets and needs, leaders should carefully 
emphasize the LDS goals: first, to provide education practitioners with the 
information they most need and want; second, to inform policy development 
and resource allocation; and, third, to ease accountability and reporting 
burdens. The culture of data collection for compliance must be overcome and 
stakeholders should focus on how the data can help them improve education 
from the bottom up as well as from the top down. 



A Model for Engagement 

An effective model for bringing stakeholders together involves creating several individual 
standing committees and stakeholder groups that hold periodic information-gathering 
meetings. The number of groups necessary to accommodate all interested parties, and the 
means of bringing participants together (in one central location, at regional sites, or via 
telephone or online conferences), will depend largely on the size of the organization, the 
geographic size of the state or district, and available resources. 

In these meetings, participants should identify their data concerns, define needs, 
and pinpoint aspects of the “here” system that need improvement. The groups may study 
other systems, learning what other agencies’ data systems look like and how they are being 
used. Such analysis can provide examples that can be used as models, starting points, or 
outcomes to avoid (see chapter 8). 

In addition to these meetings, consider using a variety of information-gathering 
strategies such as in-person and virtual focus groups, interviews, roundtable discussions, 
or paper or online surveys (Wilson and Nunn 2007). Also, take advantage of already 
established groups that may be able to help define LDS requirements and facilitate 
communication. If there are relevant task forces, working groups, or data-user groups 
already in existence, for example, ask them to carve out some time for LDS discussions. 

A representative from each stakeholder group should also serve on a core committee 
that should meet frequently to share findings from each group’s meetings. This core 
committee should play a continuing role in overseeing and facilitating communication 
about the LDS planning process, and in fostering a “living system” by ensuring ongoing 
feedback on how to design and improve the project so it meets stakeholder needs. 

Innovation requires everyone to step out of the box and consider how the system 
can work for them and what types of information would help them be more successful in 
their jobs. Participants should be encouraged to be bold and creative in their suggestions. 
Create a collaborative environment where all stakeholders feel comfortable contributing as 
equals. 



22 



Traveling Through Time: The Forum Guide to Longitudinal Data Systems 




Who to Engage 

The stakeholders invited to contribute to the early design process should vary in terms of 
their interest in the project as well as their expertise, responsibilities, geography, and the 
ways in which the system can benefit them. Include insiders familiar with education data 
and the workings of the agency, as well as outsiders who can provide a fresh perspective. 
Invite both the tech savvy and those who know the business end of the enterprise. In 
addition, involve those who collect and provide the data as well as those who use them. 

Stakeholder engagement should also acknowledge the need for a change in “data 
culture” in the education community. While data systems have historically been built 
primarily for compliance, the recent shift of emphasis towards using data to inform 
decisionmaking, improve educational strategies, and enhance student learning requires that 
these systems take local educators’ needs into consideration. For these reasons, education 
agencies should also be sure to include, in addition to state-level personnel, ample 
representation from schools and districts in the planning process. 

Contact other agencies, organizations, or other potential stakeholders who may be 
interested in the system and invite them to participate. Include as many groups as possible 
at first, and let participants decide whether to continue attending meetings. This will give 
everyone an introduction to the project and allow anyone with a special interest in the 
endeavor to become more involved. A word of caution: while it is beneficial to include 
a broad collection of stakeholders through these planning stages, there is such a thing 
as “too much” input and such inclusiveness might risk hampering the project. Setting 
goals, ground mles, and strategies early on for handling ideas efficiently will help keep the 
process moving and on track. 

Table 1 lists the types of stakeholders that might be enlisted in the planning stages of 
the LDS project. 
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Table 1 . Stakeholders who might be involved in LDS planning 



Internal 


■ Elected officials 


■ Secretaries 


stakeholder 


■ Legislative and governor's staff 


■ School counselors 


groups (state, 


■ Governing boards 


■ Librarians/media specialists 


district, and 


■ State education agency program coordinators 


■ Program area experts 


school) 


■ District superintendents and assistant 


• Special education directors 




superintendents 


• English language learner program 




■ Chief information officers 


directors 




■ Public information officers 


• Title 1 coordinators 




■ Local accountability officers 


• Title III coordinators 




■ District-level data stewards 


•Gifted and talented education 




■ Content supervisors 


coordinators 




■ Human resources staff 


■ Information technology staff 




■ Early learning coordinators 


• Project managers 




■ Guidance services directors 


• Hardware engineers 




■ Curriculum/instruction staff 


• Software developers 




■ Career technical/adult learning staff 


• Network engineers 




■ Teacher certification staff 


• Database designers 




■ School administrators (principals and 


■ Graphic display experts 




directors) 






■ Teachers 






■ Registrars 




External 


■ Advocacy groups 


■ PTA representatives 


stakeholder 


■ Children's services 


■ Parents 


groups 


■ School board members 


■ Media/press 




■ Teacher retirement board 


■ Researchers 




■ Institutions of higher education 


■ Community members/the "public" 




■ Business and industry 


■ Vendors 




■ Support organizations 


■ Other state agency representatives (e.g.. 




■ Union representatives 


Department of Labor) 
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Chapter 5 




W00^ 


Building State-District 




Relationships 




hy are state-district relationships so important, especially in the case of LDS 
development? Why are such relationships beneficial to both states and 
districts? How does a good relationship improve the quality and usefulness of the LDS? 
How can these relationships be forged and fostered? How can states get districts to buy 
into the process? This section offers best practices and insight from the states on these 



issues. 



Building Bridges and Gaining LEA Buy-in 

As discussed earlier, LDS development should involve all stakeholders (see chapter 4). 
Districts are an important part of this group and their engagement is vitally important 
to the creation of a successful, statewide LDS. In some situations, a culture shift may be 
needed to move to a more inclusive, collaborative environment. A strong relationship 
between a state and its districts offers many advantages in the development and use of 
LDSs, whether the systems are statewide or built locally to serve district or regional needs. 
Too often, the relationships between states and districts are very limited, and collaboration 
is an unfamiliar, even unwelcome idea. Communication barriers between these levels 
often have serious consequences, including mistrust or frustration, poor data quality, 
misunderstandings, or the establishment of unrealistic or unachievable requirements. 

The benefits of a good district-state relationship flow both ways. State agencies 
developing statewide LDSs can gain tremendous insight from local leaders, who can 
benefit from sharing their knowledge by getting a system that better meets their needs and 
makes their work more efficient and effective. Districts interested in building their own, 
local LDS may learn from the state. And the opposite may be true: local districts may have 
a more sophisticated system than the state’s and can share their experiences with their state 
counterparts. In either case, state and local agencies should work together to build a single 
system, or systems that will work together, to serve common goals, facilitate simple data 
transfer, and create only one version of the “truth.” (DQC 2006a) 
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/ \ Keeping districts in mind: 

J ] Considerations for state and regional 
y agencies building an LDS 



■ Consider the varying needs of your districts. Be flexible enough to serve both large 
and small districts, as well as districts with varying degrees of experience with data 
systems. 

■ Consider the extra burden being placed on districts to report data elements. 

■ Involve districts in the entire process. Find out what local leaders need from an LDS. 

• What questions need to be answered at the district-, school-, and classroom- 
levels? What data are needed to answer them? 

• What access is needed at the district-, school-, and classroom-levels? 

• What reports and tools will help local staff do their jobs better? 

■ Consider interoperability with existing systems in districts in terms of standards and 
common definitions. 

■ Consider the possibility of providing added value back to districts as incentive to use 
the system (for example, combining the state-hosted student information system with 
the LDS). 

■ How will the LDS address the need to collect and analyze local assessment data over 
time? (These assessments can be unique to schools and districts.) 

■ How will districts exchange student records (for example, release to a transfer 
student's new district)? 



With a statewide LDS project, states should think of districts as partners, rather than 
as customers. As such, the LDS should be conceptualized and developed not as something 
that districts simply need to comply with, but as a valuable tool that will benefit both the 
state and the districts. The state should think of districts not only as providers of data, 
but also as users who will benefit from data sharing and access. And conversely, districts 
should not think of the state only as a collector of data, but also as a provider of useful 
information. This sort of state-district partnership can be cultivated in several ways. 

Engage LEAs early 

Ideally, local education agencies (LEA) will buy into the LDS development process because 
they see its value, and their involvement will be voluntary. State communication with 
districts about the LDS should be framed in terms of its value to the districts, and local 
engagement should be encouraged as early in the process as possible. 

For example, district representatives should be included from the start in committees 
or working groups focused on LDS oversight and development. Small states may seek 
to include all districts in this process, while larger ones might choose a sample group of 
diverse, representative districts. In this process, states should try to gauge districts in terms 
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of expertise and the time they will have to help with the project. States may also involve 
districts in the grant-writing process or the RFP committee, both to get them engaged and 
to gain input. District representatives should also be involved in the state’s data governance 
process. In terms of data collection, for example, when a state proposes the collection of a 
new data element, participating LEA representatives will be able to offer input. The state 
may also ask all of its districts for feedback. Ample time should be allowed for response, 
perhaps up to a year prior to the collection of the new element. This way, the LEAs will 
know what is coming well in advance, and their feedback can be incorporated to make data 
collection as smooth as possible. 

Moreover, ask local representatives what kinds of questions they want answered 
(see chapter 5 of Book One: What is an LDS?). Additionally, how do they want the data 
returned to them? What information will most alleviate their pain points? Along the way, 
find out what the districts think of certain aspects of the system. For instance, in the portal 
or business intelligence tool development phase of the project, what types of tools will be 
most useful? Flow can certain reports be made more useful or user friendly? Flashy tools 
are nice, but simple low-tech improvements that save time are often hugely appreciated 
and can strengthen support for the project. 

Respond to LEA feedback with action 

Many states cite the importance of not only soliciting and considering LEA feedback, but 
also responding with action. For instance, if districts say some areas of the data collection 
are problematic, the state might seek ways to resolve the issue by adjusting the collection, 
making it optional, or even eliminating it if necessary. Districts should not feel ignored; 
this can lead to alienation and even cynicism and mistrust. However, while state agencies 
should be responsive to districts, they should also strike a balance and make only promises 
they can keep, offering a realistic view of what can be accomplished in both the short- and 
long-terms. All parties will benefit from an open and frank relationship. 



Return the data to the district 

Districts submit large quantities of data to states. This responsibility can be burdensome 
and may seem without reward. State agencies should strive to quickly return data to the 
districts in a useful form. If the districts are able to review and use the data for analysis, or 
see them in reports that show comparisons among schools and districts within the state, 
this reinforces their importance as more than just a troublesome requirement. Showing 
the districts how the data are used, and giving them a chance to use the information for 
their own needs, is an excellent incentive for cooperation from the districts, encouraging 
them to devote more energy to submitting high quality data. States can enhance this 
incentivation by seeking to collect data that the districts find most useful. 

Maintain district engagement 

District involvement should continue throughout the development process. Some 
states have brought some or all districts into the product testing phase by conducting 
pilot studies with them. Districts get a chance to try the system and give feedback to 
the state. One large state, for instance, included 10 percent of its districts in one such 
study. Continued engagement should also be sought through regular communications, 
as well as through training and feedback sessions that cover system uses and benefits. 
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In such training, district staff can be shown why it is vital they provide the necessary 
data accurately and on time. Some states have also strengthened their relationships with 
districts through financial means. According to the Data Quality Campaign (DQC 2006b), 
providing funds, perhaps by earmarking a percentage of state funds for districts to support 
infrastmcture development, has been a source of good will. In some cases, when buy-in is 
not forthcoming and districts are resistant, states may seek ways to incentivize data quality 
and timely submission of data. One approach that has been considered is tying funding to 
data quality and timely submission. Other states have had high-level executives send letters 
to districts stressing the data’s importance. 

Strategies to facilitate state-district communication 




LVS Loire/. 

Lever aplnp past partnerships 

In the past, the states districts were 
grouped into regions, with a regional 
department servicinp districts in each 
rep ion After budpet cuts, many of the 
staff in regional departments were moved 
to the state education apency. Since 
then, the state has taken advantape of 
relationships that were fostered by the 
repional service centers. Communication 
continues between the same personnel at 
the state and the districts they served, and 
these relationships continue to benefit the 
system. 
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Having existing relationships between the state and districts prior to the start of an LDS 
project is, of course, better than starting from scratch. Some state agencies have program 
area staff who already have close working relationships with the districts. They have 
leveraged these relationships in building the different pieces of the system in order to 
meet the needs of the program areas and the districts. These staff members now serve as 
intermediaries with the districts. 

Such history may be ideal, but many states and districts do not have these working 
ties, and any connections they do have can stand to be strengthened. States pursue 
various strategies to bridge this divide, but regular communication is fundamental. Some 
periodically send a newsletter to the districts, discussing specific data issues and reviewing 
progress and successes in the state’s LDS development. Others hold regular conference 
calls, online sessions, and face-to-face meetings with district staff. Small states should 
consider face-to-face meetings with districts, while larger ones may find it more feasible 
to work with regions rather than with many individual districts. Districts also vary in their 
capacity to participate in such efforts, both in terms of expertise and time, so states should 
try to assess their districts and be flexible. 

Communication with staff from all levels should be sought so that a consistent view 
of what is happening in the state is shared by everyone. The least informed groups can be 
the biggest challenge because they do not understand the issues involved. Thus, educating 
these groups is vital. In general, educating as many people as possible is essential because a 
single individual may not disseminate information to any of his or her colleagues. 

Districts should also be represented on committees, advisory boards, and working 
groups; this will help spread knowledge and facilitate communication about data issues, as 
well as build relationships among staff and other stakeholders. States might also facilitate 
communication by making it easier to contact staff members. One state, for instance, is 
creating a data area-specific directory of district and state staff that will make it easier for 
the different agencies to communicate with one another. The directory will also identify 
which staff are in charge of which data areas, allowing questions to be directed to the right 
people and increasing efficient communication. 

Another option is to use a third party to bring districts and the state together. One 
state used a consulting firm to help forge and strengthen its SEA/LEA relationships. 
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Self-Assessment: you Are Here, 
But...Where Exactly is That? 



hat is your current, or “here,” data system? The answer might seem obvious 
since, after all, this is the current operations of the organization— the 
everyday reality. But to get the LDS project off to a good start, you and your stakeholders 
should step back and carefully depict your system environment and capabilities. The 
results may surprise you. 

Invite representatives from a wide range of stakeholder groups (see chapter 4) to look 
at the organization’s current data system and data use practices. Though some technical 
staff should be involved, an understanding of the nuts and bolts is not required. On 
the contrary, the most important input in self-assessment will come from those who are 
involved with the day-to-day business operations and goals of the organization. 

Self-assessments can be carried out in several ways, such as through an LDS 
steering committee, advisory board, or working group; personal or group interviews with 
stakeholders; written questionnaires; and focus groups. Look at what system components 
and functionalities exist currently and what developments are underway. Table 2 offers 
questions to be answered in the self-assessment process. 

Table 2. Self-assessment sample questions 



■ Do you have a data collection system? Is it web-based? 

■ How do you collect the data (via paper, electronic transfer, etc.)? How often are they 
collected and updated? What is the path of data collection (e.g., from schools to districts 
to state)? 

■ How granular are your data (individual- or aggregate-level)? 

■ Do you have a unique student identifier system? Can you use it to match records across 
databases? It not, do you use Social Security numbers? In which databases are these 
identifiers used as the primary ID? 

■ Do you have a unique teacher identifier system? 



What does your 
data system look 
like? 
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What data do 
you collect? 



How is data 
quality ensured? 



■ Are student and teacher data linked? If so, how (individual IDs, classroom IDs, etc.)? 

■ Are your data linked across years? 

■ Are your data integrated with postsecondary, workforce, social services, or other data outside 
of K-l 2? If so, how often is a match-rate analysis conducted? How good is your match rate? 

■ Are the data linked across state borders? 

■ Do you use electronic transcripts to share student information? 

■ Are your systems interoperable? 

■ Do you have a central data warehouse or do programs use individual silos? 

■ What infrastructure and technology support the system (servers, software, etc.)? 

■ What parts of the system are run in-house? By a vendor? 

■ How are data secured? 



■ What data do you collect on students (enrollment, demographics, test scores and information 
on untested students, program participation, course completion, graduation, etc.)? 

■ Are these data matched for students from year to year? 

■ How often are students tested in each subject and can testing data show annual growth for 
students in any subject? 

■ What data do you collect on teachers (certification, professional development. Highly 
Qualified, salary, etc.)? 

■ What other data do you collect on the education system (financial, facilities, etc.)? At what 
levels do you collect these data (e.g., school, district)? 

■ When are these data collected and who provides them? (Catalog all current and planned 
data collections.) 



■ What types of quality assurance processes and audits are utilized to ensure data quality? 

■ Are business rules in place? 

■ Do automated data edit processes ensure compliance with the business rules? 

■ What governance structure ensures data quality? 

■ Are common course codes used? Is there a central, authoritative data dictionary? 

■ Does a data model depict the data environment? 

■ Is a staff training program in place at the local level to improve data quality? 
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How are your 
data used? 



i How are data transmitted to the state or federal government? 
i What reports are produced with the data? 
i How are these reports made available to users? 
i What ad hoc querying is available? 

i Is access to data role-based? Who is allowed access to what data? 

i How are the data presented to users (spreadsheets, web-based analysis tools, digital 
dashboard, etc.)? 

i What does staff do with the data (federal and state reporting, program performance 
monitoring, student tracking, data-driven decisionmaking, etc.)? 

i What types of professional development are provided to help staff access, analyze, and 
interpret the data? 



What other 
factors affect 
your data? 



i Do any federal or state laws and regulations control the collection and use of individual 
student data and protect the privacy of student records? 

i Do any federal or state laws prohibit or mandate linkages between P-1 2 data systems and 
postsecondary or other outside databases? 

i Do any federal or state laws or regulations require the data system to have certain 
components? 

i What is the culture of your organization in terms of data use, data 
collaboration? 



The next chapter introduces enterprise architecture, which is a process used to 
systematically conduct both self- and needs-assessments, as well as to guide efficient and 
effective system development thereafter. Whether an organization follows this rigorous or 
a less formal process, findings from the self-assessment should be carefully documented 
before moving to the next stage of information gathering and planning: needs assessment. 
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nterprise architecture (EA) is a planning and analysis tool that can help 
education agencies through the self- and needs-assessment stages of the LDS project (see 
chapters 6 and 8). Various complex definitions of EA are available, but this guide follows 
the Microsoft Corporation’s description of enterprise architecture as “a conceptual tool 
that assists organizations with the understanding of their own stmcture and the way 
they work. It provides a map of the enterprise and is a route planner for business and 
technology change.” 




Enterprise architecture is "a conceptual tool that assists 
organizations with the understanding of their own structure and the 
way they work. It provides a map of the enterprise and is a route 
planner for business and technology change." 

(Source: Microsoft Corporation) 



For the purposes of LDS development, the education agency and all of its parts 
make up the “enterprise.” The “architecture” is both the process of describing, and the 
description of, “the fundamental organization of a system, embodied in its components, 
their relationships to each other and the environment, and the principles governing its 
design and evolution” (IEEE 2000). Viewed as a process, EA identifies the mission and 
goals of an organization and the applications, technology, data, relationships, and other 
resources it uses to accomplish its work. As a description, EA documents the organization’s 
findings in the form of a top-level, low-detail blueprint that can be used to efficiently guide 
LDS development and maintenance. Though EA focuses largely on technology and the 
organization of data, its main function is to clarify the technological nuts and bolts so that 
technology and business needs can be better aligned. 

The EA should first describe the current system. The architecture exists whether you 
describe it or not, but depicting and documenting the “here” gives staff a better grasp of 
how the agency and its information system work. This depiction, at least at first, should 
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focus on high-level business operations rather than get mired in details. Then an EA 
of the future, ideal system can be created. With these blueprints, an agency will be in a 
better position to make decisions about how the enterprise needs to be modified to meet 
evolving goals (Aden 2008). The EA process also identifies who has authority over different 
system components, and who has been assigned responsibility for certain activities. EA will 
therefore be beneficial in both planning and governing the LDS. 

Figure 4 divides EA into five areas or perspectives, drawing focus to increasingly 
detailed aspects of the enterprise. 



Figure 4. Enterprise architecture in five parts 





Business 


What is my business? 




Information 


What information do 1 need? 


n 


Applications 


How is that information presented? 




Data 


What are the data components? 




Technology 


What technology supports that data? 



While many details need to be determined as organizations drill down into the 
EA process, some high-level questions will help in early LDS planning and are presented 
below. A variety of stakeholders should be involved in answering these questions about 
what the system looks like and the purposes it serves. 

Business architecture 

> What is our business? 

> Why does our business exist? What is its mission? What does it accomplish? 

> How do we do what we do? What are our core processes? Who do we serve? 

> How are we organized? How do people and processes interact to do what we do? 

> What are the strengths of our enterprise? What do we do well and very well? 

> What are our weaknesses or failures? What have we learned from these failures? 

> How will our business change in the future? What are our growth challenges? 

Information architecture 

> What information do we need? 

> What decisions do we make? What information do we need to make each decision? 

> What are the components of that information? How do we obtain each part? 

> Where does that information originate? Who creates it? What is its quality? 

> What information is needed to create the products our business produces? 

> Is any of the information highly sensitive? How is that information protected? 

> Is there information we do not have that could be valuable to our business? 
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Applications architecture 

> How is that information presented? 

> What automated services support our business processes? 

> How do our applications interact and depend on one another? 

> How are data presented to users? 

> How do our applications link various staff within the organization? With the outside world? 

> How do our applications help us transform data into information? 

> How do our applications serve different groups to achieve common business objectives? 

> Do we have plans for developing new applications and revising old ones to meet our goals? 
(Microsoft) 



Data architecture 

> What are the data components? 

> What are the sources of our data? 

> How are data managed? What business rules and quality assurance procedures are in place? 

> Do we have a data model? 

> What metadata do we maintain? Is a system in place to manage these metadata? 

> Do we have an authoritative, agencywide data dictionary? 



Technology architecture 

> What technology supports our data? 

> What technology standards and services are used to accomplish our mission? (Microsoft) 

> What technologies are used to collect and maintain our data? 

> What technologies protect our data? 

> What technologies provide access to our data? 

> What technology expertise is needed to support this effort? 



Developing an enterprise architecture is most beneficial before or during the planning 
and analysis stages of the LDS development process. But it is never too late, and great insight 
can be gained from the process at any point throughout the LDS’s life cycle. 
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M 



any states jump head first into developing or purchasing an LDS (or 
certain components) before spending time to think about exactly why 
they are doing so, what their stakeholders need, and what it will take to get the job done. 
Experience suggests that successful LDS development is roughly 80 percent planning and 
only about 20 percent building/" Careful planning can make the difference between a 
clumsy belly flop and a graceful dive into system development. 



During the needs assessment phase, policymakers should establish the business 
justification for what kind of LDS they want and why. In addition, stakeholders should 
define their requirements so that the resulting system reflects their needs. Needs assessment 
is also another step in creating early buy-in for the project, getting everyone— both internal 
and external stakeholders who use, or would like to use, education data— involved in 
creating a vision for a system that they will one day find useful. 



80% Planning 




Why Do You Want an LDS? 

Before the developers start defining requirements and building the system, some fundamental 
questions should be answered about why the system is being built and what it will do for 
the education community. Early on, decisionmakers should ask themselves the following 
questions: 



0 * 

0 * 



“Why do we want an LDS?” 

“What are our ultimate goals for the system?” 



"'This statement refers to initial development of the system, rather than the entire life cycle of the system, which will 
include ongoing maintenance of the system, substantial communication, change management, and end-user support. 
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Preferably, the answers will not be: 



0 

0 



“To check off the 12 elements of the America COMPETES Act.” 
“To keep up with other agencies.” 



Better answers are: 




“To improve instruction by helping teachers identify student needs, discover and 
practice the most effective teaching strategies, and tailor instruction; by helping 
administrators target teachers for professional development; and by helping 
researchers conduct more informative studies to identify effective strategies.” 




“To inform policy and resource allocation decisions at the state and local levels 
with better information, and to help state program staff target district and school 
improvement needs.” 




“To calculate academic growth; and follow students and maintain their academic 
histories as they advance to higher grade levels, transfer across districts within the 
state, drop out or transfer to private school or another state and come back into the 
system, and move into higher education.” 




“To track staff and maintain their professional histories as they enter and progress 
through teacher preparation programs, receive professional development, and 
transfer among schools.” 




“To streamline operations and improve data quality by automating processes such as 
data entry and loading, making data collection more efficient; by helping state staff 
produce federally- and state-mandated reports; and by conforming to broadly shared 
data standards.” 




What Do You Need from an LDS? 

Once decisionmakers and other stakeholders are familiar with the LDS concept (see Book 
One: What is an LDS?), the benefits it can provide (see chapter 5 of Book One) what their 
current system looks like (see chapters 6, 7, and 9), and why they need such a system, they 
can create a vision of what they want their own LDS to do and what functionalities they 
want it to have. 

Determine system requirements early so that expectations can be adjusted for 
everyone involved in the development process. Establishing goals and expectations for 
the system is also a good practice. This will help focus system design and keep IT staff, 
vendors, legislators, and others on the same page in terms of what needs to be done and 
what will actually be available in the future. 

Everyone wants to improve student achievement, but the important question is, 
“How do you want to do it?” Policymakers and other stakeholders need to determine 
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Key questions for key audiences 



0 - 

0 * 

0 * 

0 * 



Ask decisionmakers: 

"What do you want and why do you want it?" 

Ask education experts: 

"What will the system do for you?" 

Ask system developers: 

"What will make the system work?" 

Ask everyone: 

"How do we know if we achieved the goal?" 

"How do we identify success and anticipate failure?" 



what functionalities they want and establish requirements for the developers. If system 
developers have clear requirements to fulfill, policymakers and other stakeholders are much 
more likely to get what they need. 

You may want a souped-up, cutting edge system that will cost a lot. Or you may just 
want a basic, no-frills economical system. Either way, you must be clear about what you 
need and what you can afford. Without careful reflection and planning, your organization 
may find itself with a poorly designed system that does not really do what it needs it to do. 




Others' LDSs 

The National Center for Analysis of Longitudinal Data in Educational 
Research (http://www.caldercenter.org) offers links to several LDS 
datasets along with descriptions of fheir source systems. The State 
Data page (http://www.caldercenter.org/research/statedata.cfm) is a 
good place to research other agencies' LDSs. 



Knowing what you do not know 

It is sometimes difficult to figure out what you need because you may not be familiar 
with what you do not have. You may not realize that certain functionalities might make 
your job easier. To get started, you might compare your current system, as depicted in 
the self-assessment, to the ideal LDS outlined in chapter 3 of Book One: What is an LDS? 
Researching other data systems or visiting an education agency with a more sophisticated 
system can also provide context and insight into available options (see NCES’s Personnel 
Exchange Network at http://ies.ed.gov/whatsnew/conferences/?id=185&cid=2). Also, review 
the software applications on the market. Consider the functionalities they offer and 
determine if they would work in your environment and meet your stakeholders’ needs. 
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When working with stakeholders, try to operate in concrete rather than in abstract 
terms. Instead of asking vague, open-ended questions about what they need, it may be 
helpful to start with questionnaires to gauge interest in certain LDS components and 
capabilities. For instance, an agency might survey staff and other stakeholders with a list of 
questions that the new system could be designed to answer. The staff could then rate the 
questions in terms of the value they think the answers would provide, thus allowing you 
to gauge stakeholder interest in a systematic fashion. And, since some stakeholders will 
inevitably be more outspoken in stating their needs in meetings and other venues, such 
surveys are a good way of leveling the playing field so prioritization of needs can be done 
fairly. 

Specific questions included in such a survey might be drawn from the stakeholders 
themselves, and the rating process could provide a basis for prioritizing stakeholder needs. 
Surveys like this might be written for different groups. For instance, teachers might be 
asked to evaluate questions about curricula or student achievement, while other staff 
might be asked to assess operational functionalities such as data entry automation and data 
sharing between databases. Based on the answers provided and the resources available, 
a state might decide which data to collect, which tools and reports to offer, and which 
capabilities to prioritize. 



Assessing what you need 




Third-party needs assessment 



You may conduct your needs assessment on your own or work with a vendor to define 
your requirements. Both approaches are common. Some agencies hire a vendor to do the 
requirements gathering, allowing an expert to figure out what the education agency and 
its stakeholders need. If your agency is allowed to enter into personal service contracts, 
you may be able to employ a vendor under a time and materials contract to do the 
needs assessment for, or with, you before the RFP is written (see chapter 1 4) and the 
development project begins. While some agencies find this approach very helpful, others 
prefer to do the needs assessment, or at least a large share of this phase, on their own 
before the vendor is brought on board. In other words, they feel the agency should know 
what it wants, rather than let a vendor determine what it wants. However, if you elect to 
have a vendor do requirements gathering for, or with, you, make sure the contractor is 
knowledgeable about your agency and about education data. And make sure stakeholders 
are being asked the right questions to elicit helpful responses. Stakeholders usually know 
what they want, but they often do not know how to articulate their needs unless asked the 
right questions. 
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To help guide discussions, develop some questions that should be asked of the various 
audiences. Some examples are listed in table 3. 

Table 3. Sample needs-assessment questions for stakeholders 

■ What information would help you improve instruction and programs, and better understand and meet students' needs? 

■ What common data requests do you receive that cannot be answered with the currently available data? 

■ What additional data would you need in order to answer your questions? How can you get those data? 

■ What database linkages would be necessary to answer your questions (both within and outside the institution)? 

■ What access to data should be provided to various types of users to facilitate easier, more effective use of the data? 

■ What tools would facilitate access to, and analysis of, the data? 

■ What kinds of additional reports would be useful to staff and outside researchers? 

■ What types of professional development would be necessary? 

■ What user support would be helpful? 

■ What business operations do you want the LDS to improve? 

■ How could data collection be improved (e.g., move to web-based system)? 

■ How could the new system help you better comply with federal and state collections (e.g., collect data elements 

required by federal or state reports that are currently not collected or able to be submitted on time)? 

■ What technology would you need to make the system work? What new technical capabilities would be necessary? 

■ What additional security measures would need to be implemented to protect the new data? 

■ What capabilities do other systems have that would potentially help your agency better achieve its goals? 

Documenting requirements in some form, such as in a needs statement, is very 
important. This is “a description of the functional needs, technical requirements, and 
security and ethical standards that need to be met by a technology solution” (National 
Forum on Education Statistics 1998). An agency should take its findings, categorize them, 
and try to boil them down to discrete needs. The resulting set of requirements can help 
guide the work of in-house system developers or help an agency find a product on the 
market that will meet its needs. If you are planning to hire a vendor (or several vendors), 
your findings will help in the creation of an RFP (see chapters 13-14). Either way, be sure 
your identified needs can be measured by tangible criteria, so that success in attaining the 
organization’s goals can be assessed during evaluation of the system later on (see chapter 15). 

After figuring out what you have and what you want, a “gap” analysis should be 
done to identify the discrepancies between “here” and “there,” and determine what work 
will be necessary to achieve the desired system. For instance, if the agency does not 
currently have a student identifier system, but the needs assessment calls for one, a “gap” 
has been found that needs to be addressed. 

Building an LDS with stakeholders’ input is an iterative process. Give it time and 
be persistent. Expect needs assessment to be ongoing, rather than completed once at the 
beginning of the project. As parts of the system are developed and go live, continue to get 
feedback to find out what stakeholders think and what new needs they might have. New 
ideas often arise from new developments. 
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Realistic LDS expectations 



When negotiating with a vendor or working with in-house staff to design your LDS, 
consider asking for these important LDS features: 

■ Unique identifier system that allows individual student achievement to be tracked yet 
protects student privacy; 

■ Interoperability with other K— 1 2 systems; 

■ Interoperability with PK and postsecondary systems; 

■ Teacher value-added data; 

■ Data warehouse (or comparable alternative); and 

■ Reporting and analysis tools with easy-to-use interface. 



I 



District difference: District-level LDS considerations 

Most of the discussions about LDSs focus on state-level systems. But state agencies are not the only 
ones building these systems — many school districts and regional agencies are as well. While fewer 
people may work on a local-level LDS, the same need exists and the same mistakes are often made. 
Below are questions and considerations for school-district staff to ponder before building their LDS. 
(Many of these are also applicable to a state-level effort.) 



When assessing the need for a district-level LDS, ask 

■ What are the existing options? 

• Can the state offer LDS services? 

• Can the region offer LDS services? 

• Is it possible to form a partnership or cooperative with other school districts in order to share 
the LDS effort? 

■ Does your district have the resources to implement and maintain an LDS? 

• Can you cover the initial costs? 

• Will you need additional staff to maintain and manage data loads and reporting? 

• Can you afford the additional costs of training staff to use the system? 

• Can you afford to train staff to make informed decisions at the classroom-, school-, and district- 
level based upon the new LDS data? 

If the decision has been made that a local LDS is necessary, then 

■ Investigate what other districts in your state are doing, as well as the state agency. 

• Ensure that whatever you do fits into the big picture and your investment will not be lost 
because the state agency requires use, or interoperability with, another unique system. 

■ Coordinate with the state agency to maximize alignment with state data standards. Consider using 
data standards such as SIF specifications to connect systems. 

■ Address quality issues at the source(s) of the data before they are integrated into the LDS. 

■ Ensure that the LDS is able to handle your district's unique local assessment data needs. 
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DATA: KNOWING WHAT YOU HAVE, 

Identifying What you Need 



O 



nee the decisionmakers and stakeholders have considered the types of 
questions an LDS can answer, the organization should assess what data it 
has and what it will need to answer the questions they deem most useful. Most agencies 
already have a large amount of data, especially if they are collecting and maintaining data 
on individual students. However, while more is not necessarily better, the data currently 
maintained may not be sufficient to answer questions stakeholders have determined are 
important. This section offers some data an education agency may need to achieve its 
goals. 



Your data may be stored in a central data warehouse, or in many separate data stores. 
Likely source systems include the 

• student information system; 

• curriculum management system; 

• program systems (e.g., Title I); 

• student transportation system; 

• food services system; 

• assessment/ accountability system; 

• human resources and teacher certification system; 

• financial system; 

• instructional management system; 

• student health system; and 

• library/media system. 



After taking an inventory of what data it has, cataloguing all current and planned 
data collections, and identifying where data items are housed (and which system is the 
authoritative source for each data element), an agency should determine if it should collect 
any additional data. 

While the education community often focuses on student traits and learning 
outcomes, tmly informative education research also requires context— the students’ learning 
opportunities and learning climate. In addition to outcomes, therefore, data users should 
look at information on the inputs and processes that contribute directly and indirectly to 
student learning. For instance, in which programs does the student participate? Who are 
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the student’s teachers? What classroom strategies are used? Are there differences in student 
learning opportunities by race, sex, and/or socioeconomic status (e.g., representation in 
special education and non-college-preparatory tracks, teacher experience levels, resources, 
expectations)? What are the local financial and hiring practices? 

Keep in mind that while individual data items about students and staff are extremely 
valuable in efforts to monitor and understand student experiences, deeper analysis will 
view certain data elements in concert (as “derived” data elements, indexes, or indicators) 
and track them longitudinally. Doing so allows data users to examine the relationships 
between various aspects of reality and illuminates the trends that occur over time, showing 
what educational inputs contribute to what kinds of results for which students. For 
instance, when evaluating the success of a particular program, researchers may look at 
more than the participating student performance on assessments. The relative effectiveness 
of a particular instmctional strategy, or strategies, may also depend on context and input 
variables such as the background and preparation of the teachers implementing the 
program, how closely the program is implemented, and the characteristics of the students 
receiving the instruction. And a fair evaluation of the program will look at a host of 
discrete measures of success in addition to year-end summative test scores. Such holistic 
methodologies that combine a range of relevant data can generate powerful guidance and 
help educators more effectively meet individual student needs. 




Using the right data architect for your LDS 



The usefulness of your LDS will be greatly affected by its data architecture. A good data 
architect can create a flexible data model from the outset that will help avoid extensive 
(and expensive) changes later on. In addition to helping the agency identify the right data 
elements, the data architect can help create a fully integrated system by defining the 
relationships among those elements at different levels: conceptual (relationships among 
major concepts), logical (in terms of a data manipulation technology such as a relational 
database or XML), and physical (in terms of a particular product and means of storage 
such as a server or disk drives). (ANSI 1 975) 

Of course, an agency may opt for an off-the-shelf, one-size-fits-all data model rather than 
building a custom solution with an in-house data architect. This approach may potentially 
reduce risk and time to implementation, but is likely to work better for districts than for 
states because of the wider variety of district-level products and the fact that districts' 
research needs are usually less extensive. While a data architect can be invaluable in 
designing a data model specific to the needs of a state, existing models can be useful if 
two conditions exist: 

■ Someone on staff has sufficient expertise in data architecture and data modeling 
to make a valid judgment that the commercial product meets the needs of the 
organization. 

■ Someone on staff or on the LDS design team is capable of making the modifications that 
will almost inevitably be needed to meet the state's specific needs. 
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Education indicators 



Many resources are available to help determine what data are needed to answer 
education questions. For more information on developing and using education indicators 
to measure status and outcomes, see 

■ Forum Guide to Education Indicators (Forum 2005) 
http://www.nces.ed.gov/forum/pub_2005802.asp 

This document is designed to help readers properly create, use, and interpret education 
indicators. It also identifies standard definitions and calculations, and warns of common 
misuses of education indicators. 

■ From Information to Insight: The Point of Indicators (ESP Solutions 2007) 
http://www.espsolutionsgroup.com/espweb/assets/files/ESP_Point_of_lndicators_ORG.pdf 

This document discusses various types of education indicators as well as education 
"indexes," which are combinations of related indicators that offer more thorough views 
of educational values and trends than single indicators can provide. It also discusses 
the selection of data elements required, and the establishment of thresholds, to indicate 
the need for action. 

■ Comparative Indicators of Education in the United States and Other G-8 Countries: 

2006 (NCES 2007) 

http://nces.ed. gov/pubsearch/pubsinfo.asp?pubid=2007006 
This report presents 20 indicators used to compare the education system of the United 
States to those of other G-8 countries. Indicators focus on population and school 
enrollment, academic performance, context for learning, expenditure for education, and 
education return, educational attainment, and income. 

■ Buried Treasure: Developing a Management Guide From Mountains of School Data 
(Wallace Foundation, 2005) 

http://www.wallacefoundation.org/sitecollectiondocuments/wfAnowledge%20center/ 

atlachmenls/pdf/buriedjreasure.pdf 

Geared towards district-level management, this report presents 7 key types of school- 
level education indicators. The authors suggest that less may be more: rather than 
developing an indicator for every need, they encourage parsimony. 

■ Study of Leading Indicators of Educational Improvement (Annenberg Institute for School 
Reform, 2008) 

http://www.annenberginstitute.org/pdf/leadingindicators.pdf 

This study looks at leading indicators used to identify early signs of academic progress 
before test scores come in. These indicators may help agencies think about what 
questions they want to explore and the data they will need to answer them. 
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When identifying new data for collection, overarching goals should be used as a 
framework for selecting new elements and education agencies should collect and store 
only those data that will benefit the enterprise. Elements that stakeholders think would 
be nice to have, but which do not lend themselves to achieving stated goals, should be 
avoided. In addition, the data collected should capture the appropriate level of detail. For 
instance, when collecting data on attendance, should you collect by day, by period, or by 
some other unit of time? If attendance by day is sufficient, an agency may not want to 
burden staff further (National Fomm on Education Statistics 2009). Also, widely accepted 
standards and definitions should be followed so that the records are consistent and 
comparable to other agencies’ data. 

Table 4 presents many of the key types of data that may be contained in a P-12 EDS 
and used for longitudinal analyses, but it is not exhaustive. Appendix C offers sources 
of more detailed and exhaustive data. Ultimately, agencies should collect all other data 
required for state and federal reports, as well as other key data necessary to answer its 
stakeholders’ questions. 



Table 4. Key data to collect for a P-12 LDS 



Student 


Personal and demographic information 


Enrollment information 


data 


• Unique student identifier 


• Campus of enrollment 


• Sex 


• Grade level 




• Date of birth 

• Race 


• Attendance/truancy data 




• Ethnicity 


Attainment information 




• Language 


• High school graduate 




• Economically disadvantaged status 


• Diploma/credential type 




• Limited English proficient (LEP) status 


• School dropout 




• Title 1 status (or schoolwide status) 


• Dropout follow-up 




• Migrant status 

• Homeless status 


• Grade progression and retention 




• Disability status 


Transcript/curriculum information 




• Parent education level 


• Course codes and descriptions 




• Truant status 

Program participation information 


• Completion grades 

• Dual enrollment courses 




• Bilingual/ESL program 


Other information domains 




• Gifted and talented program 


• Student health and nutrition 




• Early childhood learning program 


• Safety and discipline 




• Individualized Education Program (IEP) 


• Transportation data (e.g., length of bus ride) 




• Special assistance program 


• Family history 

• Library records (e.g., books checked out) 




Performance information 


• Meal data 




• Assessments (summative, formative, and interim) 


• Perceptions data (e.g., student-teacher 




• Untested student records 


relationships, school climate) 




• College readiness data (AP, SAT, and ACT scores) 


• Fidelity of implementation (of 




• Grades 

• Credits earned 

• Awards (e.g., diplomas) 

• Displaced status 


programs and strategies) 
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Teacher and 


Personal and demographic information 


Professional development information 


staff data 


• Unique teacher identifier 


• Professional development training (e.g., record 


• Date of birth 


of in-service credits, type of training) 




• Sex 

• Ethnicity 


• Hours of professional development 




• Race 


Personnel information 

• School identifier 




Qualifications information 


• Job/subject assignment(s) (e.g., teacher. 




• Years of experience (by location) 


librarian) 




• College attended/certifying institution 


• Program assignment (e.g.. Special Education) 




• Highest degree earned 


• Position title 




• Academic major and minor 


• Position codes 




• Highly qualified status 


• Schedules: grade/course/period taught 




• Graduation (with dates) 


• Compensation (e.g., salary, benefits. 




• Certificates (with dates) 


supplemental contracts) 




• Licenses 


• Employment status (e.g., full-time equivalency; 




• Endorsements 


start/retirement/leave dates) 




• Staff assessment results (e.g., subject 


• Time spent on administrative duties 




knowledge test scores) 


• Tenure 

• Mobility and attrition data 


School 


Finance information 


District demographic information 


system data 


• Revenues and expenditures 


• School size 


• Salaries and benefits 


• Class size 

• School safety 




Facilities and technology information 

• Building identifiers 


• AYP 




• Building area and space utilization 


Community demographic information 




• Building condition 


• Locale 




• Classroom type (e.g., conventional, distance 


• Adult education levels 




learning) 


• Income single parent households 

• Property values 




Organizational information 

• School 

• Accreditation 

• Relationship between schools 

• District- and school-level directory data 


• Labor force data 



Sources: EIMAC 2007a and 2008, DQC 2009, NCEE 2007, Davis 2008, Center for Strengthening the Teaching 
Profession 1 994, Berry et al. 2007, and Nunn et al. 2006. Refer to appendix A for a complete list of references. 



Book Two of Four: Planning and Developing an LDS 



49 




Some Critical "Abilities": 

INTEROPERABILITY AND PORTABILITY 



b ust as monetary standards simplify marketplace transactions, exchanging 
education data between systems and applications is difficult without data 
standards. Without interoperability— the quick and easy transfer of data between systems 
via a common set of data definitions, codes, and technical software standards— resource 
exchange is laborious and taxing. In other words, whether resources are being exchanged in 
the marketplace, or data are being transferred among data systems, shared standards allow 
easy and reliable transactions. 




Interoperability is the quick and easy transfer of data between 
systems via a common set of data standards (definitions, codes, 
and technical specifications). 



An interoperable system is “an environment in which diverse data systems 
seamlessly exchange information with little or no additional effort” (Collins and Fmth 
2007). Use of widely accepted technical specifications based on common definitions and 
codes facilitates this kind of environment and allows information to be easily and safely 
shared among numerous systems and applications regardless of the platform or vendor. 
The standards offered by the Schools Interoperability Framework (SIF) Association are 
perhaps the most commonly used, though some states and districts have achieved or are 
exploring interoperability through other means. While many states have implemented 
or are pursuing interoperability, and the federal government strongly encourages the 
establishment of integrated and interoperable data systems, the majority of education 
systems are still working with numerous isolated applications."' This reality costs countless 
staff hours and resources, and limits education staffs ability to effectively use the data 
collected. 




Interoperability, by 
allowing us to link 
and easily access 
previously isolated 
data sets, lets us 
easily utilize a fuller 
range of data about 
student and staff 
experiences. 



*The seventh action step of the National Education Technology Plan (USED 2004) states that “integrated, interoperable 
data systems are the key to better allocation of resources, greater management efficiency, and online and technology- 
based assessments of student performance that empower educators to transform teaching and personalize instruction.” 
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An LDS should allow for the timely and simple exchange of data between 
applications in and among schools, districts, states, and other education institutions; as 
well as agencies and organizations outside the education system. By ensuring compatibility, 
interoperability opens the door to vast quantities of longitudinal data that may otherwise 
be too laborious to acquire. These diverse data allow us to explore questions previously 
difficult to answer due to an inability to link data from various sources— sources that 
illuminate many dimensions of students’ lives. In this sense, interoperability allows us to 
easily view a more complete and accurate picture than is possible using only fragmented 
sets of data. 

Effective use of data requires information from various sources and applications 
to work together to enable easy analysis and reporting. Data should enter the system at 
a single point; each data element should have a single, authoritative “source of truth”; 
the various applications should share data; and the data should be usable many times 
and for many purposes. This will free staff from entering information into each of the 
systems (enrollment, library, school lunch, etc.). By eliminating the need for redundant 
effort, interoperability also improves data quality by decreasing the risk for data entry 
errors, and saves time for activities other than tedious data entry and management. Staff 
will, therefore, have more time to offer better services to students, focus on teaching, and 
improve student achievement through more effective and timely analysis and data-driven 
decisionmaking. (Schools Interoperability Framework Association 2006) 



Achieving Interoperability 

How does a state or local education agency achieve interoperability using, and based 
upon, education technology standards? The shift to interoperability requires the support 
of many agency staff, including decisionmakers, data system managers, technical staff, 
program areas, and the end users of the system. This is an overall culture change in the 
organization in terms of the way data are collected, viewed, used, and shared. By bringing 
together the different players in the agency to discuss and answer key, guiding questions, 
you can begin to form a team and garner the support needed to make the project a success. 
Without support across the education agency, the chances of succeeding will diminish 
because, while interoperability is a great enabler, it is also viewed with skepticism because it 
is change that is being enabled. 

When deciding to implement an interoperable solution, an organization will need to 
address several key questions, such as: 

> What is the ultimate goal of the interoperability effort? 

> Will this project connect systems from multiple local levels to gain a 
comprehensive view? 

> What are the data you are trying to share and why? 

> Are the data compatible? 

> What standards protocols will be used? 

> Will the project be developed and implemented in-house or will a vendor direct 
the process? 

> Will the existing software applications and data stmctures be repurposed or will 
new ones be purchased or developed? 



52 



Traveling Through Time: The Forum Guide to Longitudinal Data Systems 




The answers to these questions will determine the scope and breadth of the project. 
As we drill into these key guiding questions, it becomes apparent why discussing and 
addressing them with the team forms the foundation for interoperability. 

“What is the ultimate goal of the interoperability effort?” 

When considering an interoperable solution, you should be able to clearly state, in a 
one- to two-sentence goal statement, what the project is about. If you go beyond a simple, 
realistic goal, you mn the risk of creating a project with a very broad scope that can be 
cumbersome to implement. 

For instance, a goal might be: 

“Our goal is to have one point of data entry to improve data quality, reduce data 
latency, and advance data entry efficiency, thus improving our ability to service our 
stakeholders and effectively make sound educational, data-driven decisions based on 
the most accurate information available.” 

“Will this project connect systems from multiple local levels to 

GAIN A COMPREHENSIVE VIEW?” 

This is the second guiding question for the group to consider, and it is perhaps the most 
important as its answer can greatly expand the reach and timeline of the project. If this 
project is for one district or state, that is one reflection. But if it is to span entities, many 
additional considerations come into play, such as which systems need to be interoperable 
and which contain the data needed, and are there multiple data collections involving 
multiple program areas? 

“What are the data we are trying to share and why?” 

These two questions of “what” and “why” go hand in hand. They will take time to analyze 
and discuss as a group in order to determine what is, or is not, comprehensive enough to 
be included in a meaningful interoperable system. 

“Will the project be developed and implemented in-house or will 

A VENDOR DIRECT THE PROCESS?” 

This question is significant to the project for several reasons. If you have the technical staff 
and capacity to take on a project of this scope in house, the cost may be greatly reduced 
but it may also increase implementation time. By the same token, working with a vendor 
whose core competence is implementing solutions may greatly reduce the implementation 
time, but it can increase the cost and the final system may also be based on a proprietary 
solution rather than a true, publicly available and widely used interoperable solution. This, 
to some degree, can be solved if you tell the vendor your project must be built on specific, 
commonly used education technology standards such as those developed by the SIF 
Association (http://sifinfo.org) or PESC (http://www.pesc.org). 

“Will the existing software applications and data structures be 

REPURPOSED OR WILL NEW ONES BE PURCHASED OR DEVELOPED?” 

When repurposing existing software applications and data structures, you will either 
use existing software that has already used, or could use, widely accepted standards. For 
example, many existing applications can be repurposed and adapted to incorporate (or 
accept) SIF standards. Alternatively, you can buy or develop new applications based on 
standards. An example would be one of the many software applications that have been 
SIF-certified. 
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Interoperability Timeline 

Different states and districts will pursue different timelines for an interoperability project. 
With the help of a vendor specializing in educational data systems, one state is working 
on a 12-month rollout timeline that includes processes such as hardware and infrastmcture 
development, training, and knowledge transfer. While this may be considered an aggressive 
timeline to many, it may be appropriate for others. Generally, districts and states plan 
a phased implementation over a period of 2 to 5 years. This may seem a long time to 
implement an interoperable system, but experience shows that this is the reality. The time 
involved should be considered especially when developing a project that incorporates 
multiple entities. 

Phases of the project based on the systems development life cycle include, but are 
not limited to, the following: 

> Planning: Data-needs discovery and analysis. 

> Design: Writing and release of requests for information, proposals, and/or 
quotation (see chapter 14). 

> Development: 

• Implementing a skeleton of connectivity including hardware. 

• Structuring and building, or buying, an LDS. 

• Develop validations and custom reporting tools. 

> Testing: Evaluating and/or piloting the system. 

> Deployment: Project roll-out, training, and knowledge transfer. 

> Maintenance: Ongoing system maintenance. 



Experiences on the Ground 

Some states have experienced pushback from their districts because of the extra workload 
and significant costs that may be associated with implementing an interoperable solution; 
similarly, some districts have experienced resistance from their schools. This resistance may 
be more likely in states where districts are especially autonomous or resistant to change. To 
overcome these obstacles, some states try to include at least several of the largest districts in 
the interoperability framework project with the hopes that this success will interest smaller 
districts that may have the most to gain from such data sharing. Other states have decided 
to temporarily leave some districts out of the interoperability framework, either because 
they are too small or their current software applications are not ready. In these cases, the 
state may allow the districts to submit and receive data as they have in the past, sometimes 
even on paper, as they wait to move to a fully interoperable system. The ultimate, real- 
world benefits to all involved should be stressed to gain buy-in for the transition to 
interoperability. 
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Portability 




Portability is the ability to exchange student record and transcript 
information electronically from system to system, across districts, 
and between P-12 and postsecondary institutes — within a state and 
across states. 

(Collins and Fruth 2007) 



Hurricane Katrina and the resulting displacement of students 
highlighted the need to be able to transfer student information, not 
just within a state but between systems around the country. Data 
portability is "the ability to exchange student record and transcript 
information electronically from system to system, across districts, 
and between P-12 and postsecondary institutes— within a state and 
across states." This option of maintaining, moving, and sharing a set 
of personal student information allows districts and states to quickly, 
easily, securely, and quite cheaply attain information on students 
who transfer in and out of their school system, and helps them to 
distinguish transfers from dropouts. 

(Collins and Fruth 2007) 



When an interoperable solution is implemented, portability and interoperability can and 
should work hand in hand. With the thought of allowing information to flow seamlessly 
or be “ported” between systems, you are enabling data portability at the level of the 
project’s scope— locally or between entities. Many refer to this in terms of the portability 
of “content” rather than of data but, viewed holistically, portability can refer to any 
large amount of data or content that needs to be shared. Interoperability also relates to 
the meaning of the data once they have been ported between systems. Some states are 
working on content delivery systems that are both interoperable and portable. Others are 
seeking the ability to move a student’s e-transcript, whole student record, or portfolio of 
work between trusted entities, using portability as well as interoperability. Portability can 
also imply the ability to exchange large amounts of data, such as a state report, packaged 
in a way that allows for easier movement within an interoperable framework. Many 
experts believe that future innovations will allow for greater levels of data portability 
and interoperability across sites. Others are proponents of storing all student transcript 
information in a central location and allowing secure, on-demand access by authorized 
parties rather than relying on other means of moving the data between education agencies 
and across operating platforms. 
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Chapter 11 



Ensuring System Sustainability: 

Staying "There" 



y definition, longitudinal data take years to amass. This means LDSs need 
to be around for a long time if they are going to fulfill their intended 
purposes and potential. The question then arises, what does it take to sustain these systems 
over the long haul? This chapter offers some basic strategies for the long-term, while the 
sections that follow focus on some specific solutions vital to winning broad and lasting 
commitment for your system, including marketing and relationship-building among the 
state and local education agencies. 



Winning Early and Lasting Support 




Generating the charter: 
Action steps 



1 . Get executives to sign off on the LDS. 

2. Establish goal(s) — articulate the business case for building the system. 

3. Identify the seed team. 

4. Develop a high-level timeline. 

5. Develop a high-level budget. 

6. Do the self-assessment; revisit charter; and re-examine the project's scope, 
budget, and timeline. 



A sustainable system has broad and deep stakeholder support, as well as long-term 
commitments of funds and staff. If the system is to be a success, strong interest in the 
project is vital and should be built by using effective marketing and outreach to gain 
early support throughout the education community (see chapter 12). This is vital because 
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legislators can provide needed resources and support. Agency executives will authorize 
system development and implementation. Staff will share responsibility for the system and 
have a stake in its success. And local administrators and teachers will use the system and 
come to rely on it for valuable information. This wide-ranging support will provide the 
foundation of a sustainable system. 

Legislative support 

Early on, the project should be promoted to legislators to explain the value of the planned 
system, establish expectations, and garner both political and financial support. Lawmakers 
can provide funding for the system; they can also pass legislation that supports the system 
by, for instance, writing its major tenets into law or mandating system compliance. Project 
leaders should determine what benefits should be emphasized to legislators and how to 
present them effectively. Whoever makes the proposal must have a broad understanding 
of the LDS and be ready to impart the right information. That is, they must understand 
what the legislators need to know— the purpose, potential benefits, and estimated costs of 
the system. In addition, they must understand what education specialists need, as well as 
basic technical details. They should make a compelling yet realistic proposal, taking care 
not to make promises that cannot be kept or set goals that cannot be reached. They should 
convey the big picture in plain language and avoid excessive detail, keeping in mind that 
their audience may not have expertise in education, let alone education data. 



Additional sources of funds 

In addition to relying on legislators for resource commitments, consider other sources of 
funding. As detailed by the Education Information Management Advisory Consortium 
(EIMAC 2007), potential sources of financial support for building and maintaining an LDS 
include 

> federal grants (e.g., Institute of Education Sciences Statewide Longitudinal Data 
Systems grants); 

> foundational support (e.g., the Bill and Melinda Gates Foundation); 

> separate program areas (assessment, Individuals with Disabilities Education Act, 
Limited English Proficient, Title I, etc.); 

> state-level technology bonds; 

> universities and higher education research institutions; and 

> business/private sources. 

Estimating costs 

According to some estimates, the development of a state-level LDS will cost around $1 
million per year for three to five years. However, there are many variables that affect how 
much an LDS will cost an education agency. Research how much other agencies have 
spent on their systems or various components, keeping in mind any differences between 
your agencies, environment, and system requirements (see chapter 14). Some important 
factors that may affect system costs are : 
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> Starting point 

What does your education agency already have? Can parts of the existing system 
be modified or will you need to start from scratch? What infrastructure will you 
continue to use with the new system? 

> Size 

How populated is your state or district? How many students and staff will the 
system follow? 

> State laws 

Do mandates require the system have certain characteristics, such as 
implementation of an interoperability solution? 






District difference 

Estimating the cost of implementing an LDS may be more difficult for 
a district than a state agency because of several factors. For instance, 
districts have a relatively limited ability to learn from each other's 
experiences due to the lack of networks and means of collaboration 
that exist among states. Also, the number of districts developing an LDS 
within a state may be limited, yet the experiences of distant districts 
may not translate well across state lines. Further complicating the 
task is the wide variation that exists among districts in terms of size, 
circumstance, and needs. However, in most cases, the cost of a district 
LDS should be substantially less than a state's. 



> Environment 

What is the existing level of communication and collaboration among districts, 
and between districts and the state? Establishing new lines of communication 
will take time and money. How uniform is your current system across districts 
and what standardization efforts will be required? 

> Data demands 

How many users will need access to the data and what security demands will this 
access entail? 

> Local costs 

If you are building a statewide system, what participation costs will be shouldered 
by the districts rather than the state? 

> Scope 

Do you want a top-of-the line system or a basic one? How, and to whom, will 
the data be made available (agency staff, teachers, students, parents, researchers, 
etc.)? What data linkages will be created (to postsecondary institutions, workforce 
organizations, other agencies within the state, etc.)? Beware of costly scope creep. 

> Procurement 

If you are building the system, what staff will work on the project and what 
additional staff or contractors will be required? If you buy, consider both vendor 
and in-house staff costs. 
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Success 

■ Define it. 

■ Measure it. 

■ Record it. 

■ Publicize it. 



> Overlapping efforts 

In building infrastructure or developing system components, savings may result 
by working towards multiple goals in a single effort. 

> Change 

Program changes, new definitions and updated standards, and the addition of 
new data elements may require programming changes and involve significant 
additional costs. 

> Training and user support 

How many staff members will need to be trained to use the system and new 
tools, do data analysis, interpret results and reports, etc.? What user support will 
be necessary? 

> Savings 

How much will the new system save? How will new efficiencies save staff time? 
What collections can be eliminated or streamlined by the new system? Weigh 
the benefits against the costs: over the long mn, the benefits should amount to 
big savings. 

> Maintenance 

The LDS will have not only startup costs, but ongoing maintenance costs as 
well. Who will maintain the system? A vendor or in-house staff? How often will 
hardware and software be updated? Who will update documentation? 

(EIMAC 2007a and EIMAC 2007b) 



Maintaining Long-Term Support 

Though LDS development is often referred to as a “project” (both in the real world 
and in this guide), it is much more than that. Whereas a project has an end, an LDS is 
ongoing and requires constant maintenance and enhancement. Implementation is just 
the beginning. Everyone involved should understand that achieving the system’s benefits 
will take hard work and several years. Maintain excitement along the way by structuring 
the project around incremental stages with frequent milestones. Each intermediary 
achievement should be announced to the education community so that the results of 
their efforts and resources are apparent (see chapter 12): patience and determination are 
important, but everyone needs to see results once in a while or faith in, and commitment 
to, the system may wane. 

A successful LDS may outlast political leaders, so bridging administrations is 
also important. Different people value education data differently. Therefore, a new 
commissioner or a new governor can change the support for the project. Be flexible 
when new leadership with new agendas comes along (DQC 2006). At the same time, a 
deep commitment to the system across the education agency and the broader education 
community, as well as being able to show some actual benefits, will help sustain the LDS 
through changes at the helm. 
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Establish early on how success will be measured. For example, a system may be 
deemed successful if it delivers high quality results within budget constraints and on time 
(see chapter 15). Document system successes such as new efficiencies and improvements in 
educational outcomes or processes, and use them in future requests for resources. Building 
a system with measurable deliverables and outcomes will make this possible. Education 
agency staff should also make an effort to win over new leaders to the importance and 
value of the LDS data. Brief them on system capabilities and equip them with data they 
need to prepare for meetings and make good decisions. One state’s new commissioner, 
for example, wanted to meet with district representatives. Agency staff provided the 
commissioner with detailed data on each district he was to visit. This had two very 
important, far-reaching effects. It informed the commissioner and proved the usefulness of 
the data; and it demonstrated to districts that the data they report is actually used, giving 
them greater incentive to submit high quality data. 
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G 



aining stakeholder buy-in for an LDS is critical for the long-term 
sustainability of the project, as well as for the effective design and 
utilization of the system. Unfortunately, developing a system, even one that will greatly 
benefit the entire organization and its stakeholders, may not on its own be enough to get 
everyone on board. Agencies often need to also “sell” the idea to gain support for the LDS, 
and many have pursued different tactics to do so. This section offers best practices to help 
your education agency face the marketing challenge. 



Who Is the Audience? 



History helps 

Take advantage of relationships that exist throughout 
the education community, especially between state, regional, and local 
education agency staff. Get the word out and keep these lines of 
communication open. 




Cast a wide net to promote the system, reaching out to stakeholders throughout the 
education community and focusing on all levels. Present the project to education agency 
executives and decisionmakers at the top, focusing on the policymaking advantages 
of the LDS. Brief them regularly on the progress made, as well as how the system can 
help achieve the organization’s goals. Buy-in at the top may also increase when the 
LDS team moves forward with the project— contacting districts or schools for data, for 
instance. In addition, take a grassroots approach to publicizing the project. Make sure that 
information and support travel up, as well as down, the organizational ladder and ensure 
that all stakeholders realize that the LDS will be critical to the organization’s day-to-day 
operations. 

Put special emphasis on the business side of the organization. While the project will 
need support from your organization’s technology team, buy-in from nontechnical staff is 
critical. Involving a wide variety of stakeholders in the design process will help everyone 



Book Two of Four: Planning and Developing an LDS 



63 






feel connected to the project and gain early buy-in (see chapter 4). Get input from program 
area staff, district representatives (for a state LDS), school administrators (particularly 
important for a district LDS), teachers, and other interested parties before designing your 
system; and let everyone know you are incorporating their feedback. What data do they 
want? What information will help them save time and be more successful? What tools will 
best suit their needs? How will privacy be ensured? (See chapter 4.) 




To hush or to holler... 



Should you keep a low profile or grab a megaphone and yell about your LDS 
from the rooftops? Education agencies have taken both paths: some have 
worked under the radar, while others have aggressively advertised their project. 
Both approaches have their advantages. While the latter course may slow 
down the development process by inviting lots of interest and involvement, the 
limited outreach approach may lead to serious problems later on when the 
agency tries to build support for the project. 



What Is the Message? 

Communicating about an LDS development project can be a major challenge. However, 
getting everyone to understand what the new system will be, and what benefits it will offer, 
is important if the project is to succeed. 

Start with a clear vision for the project. Communicate it consistently, and be 
prepared to repeat your message multiple times. Frame the LDS around organizational 
goals and how the new system will better equip everyone to achieve them (ESP Solutions 
2007). Get the word out: 

> What will the new system be? How will it be different from the current system? 

> Why is the system being developed? How will it benefit the education 
community? Stress streamlined daily operations, time and money savings through 
increased efficiency, quicker access to data and better informed decisionmaking, 
and improved services. Make sure the LDS is not perceived as a burden. (See 
chapter 5 of Book One: What is an LDS?) 

> How will the system better equip the organization to meet its goals? 

> How will the data be protected? 

> How will the culture of the educational community need to change in terms of 
the way people think about and use data? 

Also provide frequent updates: 

> Who is backing the project? What support and resources have been won (governor 
buy-in, grants awarded, etc.)? 
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> What progress has been made toward system goals? Which benefits and 
functionalities have already been achieved, and which will materialize in the future? 

> What is the timeline for the system? Be realistic in your estimates so that 
expectations are appropriate. Highlight planned milestones and gain support for 
reaching them. 



Converting the LDS nonbelievers 

While many will get on board early without much convincing, there will probably be 
individuals who are not so willing to embrace the project. For instance, many schools and 
districts do not see data systems as critical to their day-to-day operations. Show them the 
value of the system. Win these people over by showing them that the LDS will improve 
their days and make their jobs easier. For example, what information will they be able to 
get through the portal? What reports will be made available to them? What data entry 
will become automated? How will greater efficiency improve the data’s timeliness and 
usefulness? How will better information equip them to make better decisions and spend 
their time and resources more effectively? Note that improvements do not always need to 
be high tech. Often, the changes that please local staff the most are simple ones that save 
them time every day. 

Explain why the system they are using and the data it contains are inadequate. For 
example, the current system may collect and store redundant and/or conflicting data in 
various silos, rather than collecting data once and storing them in a centralized, integrated, 
and authoritative data storage facility. Data entry may be laborious, or getting information 
back in a timely manner may be difficult. Then show how the new system will address 
these issues and improve their data. For example, if your development project involves the 
construction of a data warehouse, identify all of the silos the LDS will replace. While the 
silos may have served the state well in the past, explain that the new data warehouse will be 
better for reasons x, y, and z. For example, how will the new system be different from— and 
better than— existing transactional or other types of historical data systems in use? What 
insights can new data linkages provide that could not be derived from a system of disparate 
silos? Ease anxieties by stressing that the system is being implemented to help staff, not to 
replace them. 

When all else fails, more coercive measures may become necessary. For instance, 
an executive such as a superintendent might send out letters or emails to noncompliant 
districts requiring them to get on board. Some states make compliance with the LDS 
mandatory. If that is the case in your state, communications can be more forceful than 
merely offering encouragement. 

This following plan has been adapted from “Strategic Communications Planning,” 
available at http://davefleet.com/2008/08/strategic-communications-planning-a-free-ebook. See 
this resource for more guidance on developing a communication plan. 
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Strategic communication plan 



Develop a communication plan early on to identify your outreach needs and carry 
out an effective public relations campaign. A strategic communication plan may 
consist of the following sections: 

Analysis 

■ Context: What has happened before? What is the relevant history? 

■ Environmental scan: What are the key factors that will affect the project's 
success? What are other jurisdictions doing? What is the legislative context? 

■ Stakeholder analysis: Who are the LDS stakeholders and what are their 
characteristics? What are their expected reactions? How can the support of 
those who react positively help mitigate the concerns of individuals who may 
resist the project? 

Planning 

■ Objectives: What clear and measurable goals do you want to achieve in your 
communications effort? To educate your stakeholders? To build support or 
create demand for the LDS project? 

■ Strategy: From a top-level perspective, how will you achieve your 
communications objectives? Should your approach be high- or low-profile? 

■ Audiences: What specific key audiences will you try to reach? What are 
their needs and interests? How will you tailor your communications to these 
various audiences and maximize the impact of your message? 

■ Announcement: Given the strategy, should you make an announcement? If 
so, how will you summarize the project in your announcement? 

■ Message: What are you trying to tell stakeholders? What is the project and 
why are you doing it? What will change as a result of the project and the new 
system? 

■ Tactics: How will you implement your strategy before, during, and after your 
project announcement (if you make one)? What modes of communication 
will you use (emails, direct mailings, speeches, meetings, training sessions, 
web conferences, presentations, websites, press releases, etc.)? Who will be 
responsible for different communication activities, and when will each take 
place? 

■ Issues: What problems might you have to overcome? Can you anticipate any 
potential confusion or pushback? How will you deal with these issues if they 
arise? 

■ Budget: What will your communication plan cost? Where will funding come 
from? Do not ignore seemingly small details. 

■ Evaluation: How will you know how well you have achieved your goals? How 
will you measure your marketing success? Will you be able to see a before- 
and-after effect of your efforts? For instance, did the project receive media 
coverage? How did stakeholder perceptions change? 
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Who Should Do the Outreach? 

When trying to get the word out, having the right people promoting the project is 
important. Identify these passionate communicators among your stakeholder groups and 
deploy them to talk with their peers. Energized district leaders, for instance, can talk or give 
presentations to other district leaders, sharing with them the benefits of the LDS and the 
status of the implementation process. They can also solicit valuable feedback. Enthusiastic 
legislators, governors, state superintendents, and other high-level leaders can also make 
a strong impression and help overcome resistance and remove political barriers. These 
high profile advocates, or “champions,” can speak at meetings or communicate through 
mailings to raise awareness and get the system on people’s radars. Alternatively, these roles 
might be filled by outside consultants who can focus exclusively on the marketing effort. 
Either way, these champions should be knowledgeable about the system and the progress 
towards its implementation. 



What Modes of Communication Should Be Used? 

While agencies have used a vast arsenal of communication techniques, a large share of the 
information about the project should be shared during internal and external stakeholder 
meetings. Presentations in these meetings or at conferences (both in-person and web-based) 
should include progress updates. Training sessions and other development activities are 
also great venues for marketing the system. Other means of communication include email 
campaigns; paper mailings such as letters, newsletters, or brochures; and press releases to 
announce the project and milestone achievements. If the media are contacted, staff must 
be ready to effectively and consistently respond to their questions and concerns if support 
is to be gained from the general public. Assign specific personnel as the go-to people 
for specific types of information on the system, and direct calls and questions to them. 
Agencies might also consider creating a web page or site dedicated to the project, to which 
stakeholders can be directed for up-to-date information. 

Agencies must be able to disseminate information quickly in a format that is 
understandable to nonspecialists. Again, have a clear message and make sure that your 
audiences understand what you are saying. Avoid miscommunication, for example, by 
limiting the use of jargon in general presentations and communications. Agencies might 
also consider creating a common glossary of terms for stakeholders to reference. 

(DQC 2006b) 



Maintaining Support 

Communications should be used to keep everyone excited throughout the project. An 
LDS development project plan that includes many short-term deliverables will lend itself to 
a successful marketing effort. Show progress, however small, by announcing achievements 
and delivering results along the way. With the completion of each deliverable, a separate 
communication can be disseminated to celebrate each little “win.” These victories can be 
advertised via the media and throughout the organization. But be careful not to promise 
anything you cannot deliver quickly. If results are not forthcoming when promised, or if 
your first success is long to come and subsequent ones infrequent, faith in the project and 
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LOS lore* Knowyour audience 
and communicate appropriately 



An agency staffer met with n group of 
super Lett endfents offer luudng sent a, half- 
dozen communications about they system. 
After sitting down, shey quickly r enlisted that 
none of them haul ay clue about they system, 
but shr couldn't blarney them. They were 
probably jaded because numerous similar 
projects haA been attempted in the past and 
failed. After the meeting, instead of writing 
summaries of the project, the staffer starter! 
to send out brief communication such as, 

“ The system redesign is really happening 
this time. Here’s Oy website that details the 
progress. ” This sawed her time and let the 
reader learn about the new system on their 
own time. 



your ability to create and maintain support may be diminished (DQC 2006b). And, in 
addition to talking to staff about the project, project managers should also acknowledge 
a job well done. A little recognition or token of appreciation for those working on the 
project will go a long way in keeping everyone motivated and invested in the project. 

Finally, the system and the data themselves are another important marketing vehicle. 
Along the way, get the data out and show stakeholders its usefulness. Further down the 
line, when parts of the system are ready for testing, pilots are also an effective means of 
winning over key stakeholders. They get districts and schools interested in the system, let 
them try it out, and give them the chance to help improve it and resolving any kinks (ESP 
Solutions 2007). If a new data mart is created, for instance, let the appropriate stakeholders 
explore it and see how it will benefit them. If an analysis tool is created for teachers or 
administrators, let them see its value and chime in on ways to make it more useful. 
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Part III 



Getting from "Here" to "There": 
Developing What you want 




hen an education agency has firmly established both a vision and support 
for the LDS, focus should turn to implementation. Now that they know 
where “there” is, agencies can depart from “here” and begin work towards the desired 
destination. LDS projects may either be planned in phases or confronted all at once, 
and states have taken both routes. Which is most appropriate for you depends on many 
factors; perhaps the most important are the agency’s available resources and the scope of 
the project. 



Part III discusses many of the decisions an education agency will have to make and 
the challenges it may face while building or purchasing a system. It also offers tips to help 
make the ride to “there” go more smoothly. 



Book Two of Four: Planning and Developing an LDS 



69 





hether an agency is building a data warehouse or adding reporting and 
analysis tools to an existing depository, someone has to develop and 
implement these products. Whether to take on these tasks within the agency or hire a 
vendor to tackle them for, or with, you is an early and cmcial decision. This chapter’s goal 
is to help you become an informed builder or consumer before you begin construction 
or hire vendors, in order to ensure you end up with a system that suits your needs and is 
worth the money. A collection of lessons learned and good practices, this section will help 
you choose, and steer you in a good direction once you have decided to either build or 
buy a solution, or perhaps pursue a combination of the two. 



Deciding Which Path to Take 




Take baby steps and celebrate each one 



Even with ample resources and support, the general consensus among experts is 
that breaking the project into manageable, small, short-term stages is the best way 
to go — one baby step at a time. 

Not only is attempting an entire LDS project at once a daunting challenge, it may 
also strain an organization if staff need to devote time to extra tasks, hurt morale 
if successes and benchmark achievements are infrequent, and ultimately lead 
to serious compromise in terms of quality. Establishing a clear, long-term plan 
and breaking the project into phases offers many advantages. Working on one 
or two goals at a time is manageable for staff; and small, short-term successes 
keep everyone excited and focused. Be sure to highlight and celebrate each "win" 
along the way. On the down side, the slower pace could make some stakeholders 
impatient and draw out the timeline for the entire project. But in the end, a better 
system will result. 
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For some states, the choice between building or buying is obvious from the outset. 

For others, much deliberation is necessary. The decision depends on many factors. For 
instance, organizational characteristics such as technical capabilities, workload capacity, 
and culture are all important considerations. Does the organization have, and will it 
continue to have, the staff and expertise needed to carry out the project and maintain 
it throughout its life cycle; or is additional staff or a vendor needed? Beyond in- 
house characteristics, organizational preferences play a role, deciding, for example, the 
fundamental question of where responsibility will reside— in-house or with a vendor? Your 
organization should also consider other questions: 

> What can we build in-house that we cannot get, or do not want to get, from a 
vendor? 

> Can we get what we want from a vendor? 

> Will we need to settle for a system that does not completely meet our needs? Or, 
will we need to spend resources to modify the product to meet our needs? 

> Do we have the technical capability to build the system on our own and keep up 
with evolving data standards (e.g., federal reporting requirements)? If so, can we 
do a better job than a vendor? 

> Can we complete the task more quickly than a vendor? Or, can we meet deadlines 
without external assistance? 

> What if we lose key staff during the project? 

> Can we reduce project risk with a vendor? 

> Can we save money by building on our own or will it be cheaper to buy? 
(Consider ongoing maintenance costs for a vendor-built system and the savings 
you might have if knowledge resides in-house.) 

> What will be the ultimate trade-off between relying on a vendor and developing 
the expertise in-house? 

> What procurement approaches have other states taken? 



Why build? 

Some states prefer to keep the job in-house. They may have great staff with the necessary 
expertise and extra resources available to direct those staff members to the new project, 
or the ability to hire additional staff to do the work. Some states believe that, if you can 
afford to do it, build the LDS in-house. This, they say, is ultimately more cost effective 
because add-ons to vendor-built systems can cost a lot of money. If you have the 
capability, keeping the knowledge and ownership “in the family” will free you from the 
potentially costly prospect of having to rely on a vendor for ongoing services. Some states 
also feel that, in general, it takes just as much effort to work with vendors to make sure 
specifications are met as it does to build applications in-house. And working with staff may 
help ensure long-term sustainability because important knowledge about the data, source 
systems, reporting needs, making modifications, and other key issues are already in-house. 
And in fact, some research findings suggest that if an agency’s capabilities are strong, or 
even just moderate, building is the better option: the organization should use its resources 
to tackle the project on its own and to further develop its staff expertise rather than bring 
in outside help. The staff s greater understanding of the organization’s culture and goals 
may, in the end, compensate for a less technical skill set (Nevo et al. 2007). 
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Why buy? 

Other states see hiring a vendor as the best way to go. This may be because they like a 
product on the market and see no need to “reinvent the wheel” when it has been used 
elsewhere with satisfactory results. States often purchase such off-the-shelf solutions and 
“brand” them— that is, putting their education agency’s logo on a store-bought solution. By 
using a vendor, the state is freed from the details and can focus on the big picture and on 
selling the system to stakeholders. 

Another reason to use a vendor is that an agency with limited resources and 
capabilities may find doing the work in-house to be unrealistic, even if it wants to. If the 
state does not have a staff member with the same level of knowledge as a vendor’s project 
manager, or the capacity to build its desired product on its own, it may need outside 
assistance. In fact, research suggests that when internal IT capabilities are weak, the support 
of a knowledgeable, external consultant can boost productivity and reduce overall costs 
(Nevo et al. 2007). In addition, state hiring regulations that limit the size of SEA staff or 
impose other constraints may also make buying a more attractive idea. These rules can 
make it difficult to hire programmers. In this case, or if the organization is having trouble 
finding or keeping the necessary in-house staff, hiring vendors may be the simpler option. 

A short timeline might also necessitate the use of vendors as the number of 
programmers, architects, business analysts, project managers, senior developers, and other 
staff they have available may allow the project to move much more quickly than it could 
if shouldered by in-house personnel. Provided your organization can find a vendor(s) and 
product(s) that will meet your stakeholders’ needs, a commercial, off-the-shelf solution 
may also offer economies of scale in terms of leveraging design, including scope of data 
elements and requirements, warehouse structure, data model, data dictionary, submission 
procedures, etc.; as well as with development, testing, enhancement, support, optimization, 
documentation, and methodology across many customers. Whatever the reason, done 
correctly, choosing to bring in a vendor can be a very rewarding and successful course of 
action. Of course, every state’s experiences will be different, and careful self- and needs 
assessments are vital whether you build or buy. 



On Building 

If your organization has decided it can handle the project without the assistance of a 
vendor, either the necessary staff are on board or new staff members will be hired. A 
central challenge your organization may face is finding and holding on to skilled technical 
staff. These experts are often drawn away from education institutions to the private sector 
for reasons such as higher pay. Many states have lost key team members in this way and 
such losses caused major setbacks to their project. Therefore, before you decide to build 
a solution, ask your organization how it will safeguard against this loss of staff and the 
knowledge they hold. If staffing is an issue, going with a vendor may be a more secure 
option. 
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On Buying 

If you have decided not to go it alone, your organization has chosen to buy a product 
and/or services from a vendor. You must now figure out what you need, what is available, 
what kind of relationship you want with your vendor, and how much you are willing to 
spend. Then you will need to write a good request for proposals (see chapter 14), interview 
some bidders, and select a vendor (or consortium of vendors). Finally, you must forge an 
effective relationship with the chosen vendor to get the product you require. 

Many agencies use outside contractors and consultants to build their LDS or various 
components of the systems. These agencies may have one in-house employee who is the 
project manager, while consultants serve as programmers and system builders. Other states 
hire project managers as part of the vendor team. State agencies without the necessary 
expertise rely more heavily on their vendors and often seek a consulting firm they will not 
need to micromanage. However, states should be somewhat wary of depending too much 
on vendors, since they could unexpectedly close their doors or scale back their services. 

States with considerable in-house knowledge may seek a relationship that gives them 
significant control over the vendor’s work. These states act as CEOs overseeing the work 
of others. In this case, vendors may be hired to do the technical and programmatic work, 
and may even participate in committees to bridge the divide between their staff and the 
in-house executive levels. 



Choosing a vendor 




States and vendors: 

Who's using who for what? 



The Data Quality Campaign (DQC) offers a table of vendors listing the number of 
states they work for and on which of the DQC's Ten Essential Elements they were 
hired to work. It can be accessed at http://www.dataqualitycampaign.org/survey/ 
vendors, and includes a line labeled "In-House" that shows the number of states 
building each of the components on their own. (Note: The table is based on 2008 
survey results.) 



Once you have written and sent out the request for proposals (see chapter 14), bids and 
proposals will come in from competing vendors. Now, of course, you have to decide 
which vendor to hire. You should use the proposals to narrow the field to a small set of 
top contenders. Keep in mind that vendors vary in many ways, including in their bidding 
practices. 

In making your decision, look beyond the vendors’ marketing to the details of what 
they can actually provide. Two core principles to follow are: know the product, and know 
the team. Rather than taking the lowest bidder, be sure the vendor can actually fulfill 
the contract, at the agreed-upon price. Look at the vendors’ references, prior work, and 
available staff to try to evaluate them. Consider their experience working with states similar 
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to yours in terms of size and student population. Will their solution accommodate your 
specific needs? Do they have enough staff to deal with possible turnover? Do they have 
the staff to adequately service all of their contracts, oversee the work, and complete quality 
deliverables on time? Or are they stretched too thin? In the end, careful selection may save 
you the costs of hiring a second vendor later on to complete the work or fix issues left by 
your original consultant. Also, as you will be spending a lot of time with this team, social 
considerations are also important. Will you like working with the project manager and 
staff? 

Many agencies hold face-to-face interviews with a set of prospective vendors. These 
meetings offer the advantage of allowing you to meet the team of people with whom you 
will actually be working. Consulting firms should give presentations, demonstrate their 
products, field questions, and be allowed a chance to ask some questions of their own. A 
review board might be created by the education agency to evaluate the various vendors 
and should include agency as well as local leaders and prospective users of the system to 
get a variety of perspectives and insight. Other agencies do not hold face-to-face vendor 
interviews. They make do with reference checks, answers to standardized questionnaires, 
and phone and online meetings. At the very least, they say you can learn a good deal about 
a vendor by the types of questions they ask. What is their level of technological knowledge 
and capability? How familiar are they with education data and their nuances? For instance, 
a vendor might have an extensive background in finance data and might give you a 
product that would work well for a financial institution, but not so well for an education 
agency. Make sure the vendor is ready for the task at hand. 

Working with a vendor 



Create a Wiki 




One state is in the planning phase of developing a wiki, a "piece of server 
software that allows users to freely create and edit web page content using 
any web browser."* This resource will be used to create a more collaborative 
environment in which the state and its vendors can work. The wiki will allow the 
state to post documents and receive feedback from vendors. After a substantial 
amount of information is accumulated, the wiki may eventually evolve into a 
user guide on the LDS. 

*Source: http://wiki.org 



Do not expect a vendor to hold your hand and guide you through the project design 
process. While many vendors have extensive experience supporting education agencies, 
others are not experts in education data or needs. In this case, many decisions will be 
left up to the client organization— you. A state or district must therefore have a complete 
understanding of what is planned in order to ensure their needs are met. Detailed planning 
must be done upfront and cannot be avoided. At the same time, the vendor team should 
have some knowledge of the education agency’s data, or be willing to learn about the 
nuances of education data (for example, the importance of striking the right balance 
between confidentiality and not eliminating too many dropouts from the dataset). If 
necessary, ensure you have the time commitment from the in-house project manager, 
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as well as the subject area experts and IT staff, to educate the vendor on source systems, 
reporting needs, validation of work, and other important issues. 

Make sure the vendor staff working on your project are up to par. Some vendors will 
give you a top-of-the-line project manager, for instance, while the technical and subject 
area staff doing most of the work may not be sufficiently qualified. Having an in-house 
staff member who is as qualified as the vendor’s project manager or at least able to evaluate 
the work being done will be particularly important in such a case. Without knowledgeable 
in-house staff who can monitor the vendor’s progress, you will be to some extent at the 
vendor’s mercy and may end up with a deficient system. You need an evaluator who 
understands the technology and can keep the project on track. The in-house project 
manager should hold frequent meetings with the vendor to obtain updates, assess progress, 
and address problems as they arise. Keeping minutes at these meetings is also good 
practice, as this helps keep everyone on track and on the same page. 

Everyone wants to get along, and many states have excellent relationships with 
their vendors. But if a project seems to be getting off track or falling behind, be firm 
and hold your vendor accountable for failing to deliver. Tmst and deference should be 
balanced against vendor management and the need to get things done on time and to your 
specifications. Some vendors tend to give clients a bare bones version of their specified 
product— meeting the requirements of your RFP but not going the extra mile to create the 
ideal result. Even if the vendor is going to give you a no-frills product, you still want a 
quality version of what you asked for and were promised. If necessary, do not be afraid to 
fire your vendor and bring someone else in if the project is not succeeding and the working 
relationship cannot be salvaged. 

Plan for knowledge transfer 

Knowledge transfer is the passage of information or expertise from one organization to 
another, or from one part of an organization to another. From the early stages of your 
vendor search, your agency should strongly consider knowledge transfer and training. 
Indeed, a knowledge transfer plan is vital and should be drawn out in advance, perhaps 
even specified in your RFP, and implemented along the way or once the project is nearing 
completion. The first step is, make sure you have enough staff in place to maintain what 
is built. Diligent documentation of everything done by the vendor, as well as by in-house 
staff, for the entire project is also crucial to efficient knowledge transfer. Additionally, 
your staff should work alongside the vendor for some time, or receive sufficient training 
from the developer to understand how the system works. Before the vendor hands off 
the project, in-house staff should be involved at least once in various processes to learn 
their structures and contents so that they have a grasp of what the LDS contains, how it is 
structured and formatted, how to load data, how to make modifications to reflect changing 
requirements, and other key issues. Additionally, the in-house responsibility must be 
clearly defined so the transition from vendor to purchaser runs smoothly. 

Some states indicate that, once the vendor’s main tasks are completed, long-term 
contracts are not always cost effective, but this depends on your situation. On the one 
hand, developing in-house expertise may be advantageous so you are not dependent on 
outside support. On the other, developing the expertise required to fully support a system 
may be cost prohibitive for your agency, and relying on a vendor may be cheaper in the 
long run. 
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Chapter 14 






Writing a strong Request 




for Proposals 



h f your organization has chosen to buy an LDS, or at least some portion of 
it, you will need to write a request for proposals (RFP) to solicit bids from 
potential vendors. This section will provide a brief introduction to effective RFP writing. 
The information in this section draws heavily on Nancy Smith’s report, Lessons Learned: 
Writing Requests for Proposals for Statewide Student Data Systems in Education. Additional 
resources in appendix C include an extensive collection of sample RFPs from state 
education agencies (SEA). 



Basic checklist of RFP development 

■ Know your needs before you write an RFP or begin development work. 

■ Follow your state's procurement procedures. 

■ Define clear, specific, and measurable requirements. 

■ Delineate critical requirements from optional enhancements. 

■ Focus on both process and outcomes. 

■ Build in safeguards to protect your agency. 



Define Your Needs First 

Before writing the RFP, you should have already completed thorough self- and needs 
assessments (if not, see chapters 6 and 8) and be ready to translate your requirements 
into an RFP that tells vendors what you want to purchase. If you are not sure of your 
requirements when you write the RFP, the bidding and development processes may 
become frustrating ordeals for everyone involved, and the completed system might not 
serve you well. If you did not separate needs assessment into a discrete activity completed 
prior to the development stage, analysts will have to define needs as they go along and 
solutions-building may be slowed down. 
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Start with an "RFI" 



Before you write an RFP, consider issuing a request for information (RFI). In 
this document, an agency presents the problems it wants to solve and asks 
vendors to propose solutions, pose questions, and make suggestions. The 
advantages of issuing an RFI are that agencies receive early feedback from 
experts on the feasibility and cogency of their planned system, as well as 
learn about the range of possible solutions available to them before they write 
their RFP. 



Take the time to discover your program area needs. How will you need to restructure 
the data for the LDS? What data will the users need? How will you need to deliver your 
data to users? How will you need to secure your data to protect student privacy? Try 
to foresee as many needs as possible to limit the number of requirements added during 
development. RFPs issued by other education agencies for similar purposes can provide 
helpful ideas (see appendix C for additional resources, including some RFP examples). 

A quality RFP contains a nearly exhaustive collection of clear and specific requirements. 
Encourage vendors to be creative, but do not leave room for misinterpretation. For 
instance, do not write “etc.” in your RFP— say what “etc.” represents. Do not ask for 
“quality”— explain how you will measure “quality.” 

Writing the RFP 

Once you have defined your needs and those of your stakeholders, the next step is to write 
a strong RFP. This will help you find the right vendor, with the right product and services 
for the right price. Some states have a standardized method of writing RFPs, others are 
less formal and prescriptive, while still others do not write RFPs at all but may still need to 
document their requirements in some form. 

The team assigned to write the RFP should include staff from various education 
agency departments. Ideally, the group should also comprise subject area experts who 
understand the content and business operations, policies and regulations, and the potential 
budget; as well as technology staff who understand the codes and bytes. District and/or 
school representatives should also be involved in defining requirements. 

There are many approaches to writing RFPs. For instance, you may write one RFP 
for the entire system, compose one RFP with multiple components and let vendors choose 
which parts they will bid on, or create a separate RFP for each system component. If you 
take the latter approach, however, do not break the project into too many parts and focus 
too closely on specific pieces of LDS functionality. If the goal is a comprehensive LDS, 
writing RLPs for many small pieces may attract specialized vendors outside of education 
while dissuading those with more comprehensive, LDS-specific products and services. 

Additionally, the RL P may be based on process or on solutions (deliverables or 
outcomes). In a process-based RFP, an education agency defines the “process” the vendor 
must follow. For instance, you might require a standardized development process based 
on the systems development life cycle, though this approach might discourage vendors 
from responding if their products have already been through a development process. 

A solutions-based RFP describes the desired results, but leaves the vendor to determine 
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the process through which they are completed. A combination of these approaches may 
be best as each may lead to potential problems. Focusing solely on process may affect 
deadlines or result in final products that do not meet agency needs. On the other hand, 
concentrating on outcomes alone may ignore important details, such as the frequency of 
collections and integration with other applications. 

Another recommended approach is to find a system that you like at another 
education agency, get a copy of that agency’s RFP, and make necessary modifications based 
on your specific needs or your agency’s RFP-writing guidelines. This will save you the 
trouble of starting the RFP from scratch. 



The Forum has more detailed information... 

... about RFP writing: 

■ Forum Unified Education Technology Suite (2005) 
http://nces. ed. gov/forum/pub_tech_suite. asp 




Anatomy of an RFP 

An RFP may be composed of the following sections: 

> Project overview and administrative section 

Consists of an overview or summary of the problem, along with administrative 
information about the expected management of the RFP. 

> Technical requirements 

Provides technical requirements and enough information to help the vendor 
understand the request and write a firm proposal. 

> Management requirements 

Specifies the expectations for managing and implementing the project. 

> Supplier/vendor qualifications and references 

Asks for vendor qualifications and references; ideally specifies the format in 
which these should be supplied. 

> Supplier’s section 

Allows vendors to include information about themselves or about the project 
that is not specifically requested. 

> Pricing section 

Specifies how the pricing information should be provided. 
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> Contract and license agreement 

Contains the purchase contract, nondisclosure agreements, and other legal 
documents. 




> Appendices 

Includes relevant information that is too bulky to include in the RFP such as 
network diagrams, outlines and requirement studies. 

(Reprinted by permission from Smith [2004], Lessons Learned: Writing Requests for Proposals 
for Statewide Student Data Systems in Education.) 

Tips for RFP writing 

Here are some pointers to keep in mind during the RFP writing process: 

> Provide background 

Tell the vendor about the state’s data environment, database development 
standards, business requirements, and other relevant information about how the 
agency does business. Without this information, vendors may make erroneous 
assumptions in their proposal and through the development process. 



> DO NOT SPECIFY THE SOLUTION 

Focus on the problems you want to solve and the existing environment in which 
the solution must work, not on a particular, perceived solution. Give vendors 
room to suggest their own solutions to those problems. 



> Define clear, concise, and measurable requirements 

Distinguish critical requirements from nice-to-have add-ons. The vendor needs 
to know which requirements to include in the bid and which are optional 
enhancements that can be priced separately. This will make it easier for vendors 
to bid, and easier for you to interpret the proposals. 

> Specify costs, or do not 

You may elect to specify the amount you will pay for a solution and have 
vendors say, “Yes, we can deliver for that price.” Or you may withhold cost 
estimates and ask vendors to make offers. One state uses the latter approach, 
but asks vendors to submit their costs in sealed envelopes. The agency staff then 
assess each proposal before opening the envelopes, judging each bid on its merits 
rather than by its price tag. Once the judging period is over, if favorite vendors’ 
prices turn out to be too high, the agency tries to negotiate a more acceptable fee. 

> See how much others paid 

When trying to assess how much a solution should cost, ask colleagues in other 
agencies for information on their costs. Consider differences between your state 
or district and theirs (student populations, varying requirements, etc.) and make 
necessary adjustments. 



> Inquire about management practices 

Ask how the vendor manages projects in terms of communication, risk 
management, methodology, quality reviews, etc. 
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> DO NOT BE TOO AGGRESSIVE IN YOUR TIMELINE 

Be realistic and try to strike a balance between quality and speed. In addition to 
deadlines for the vendor, consider your agency’s end of the bargain. What will 
you need to do or provide, and how much time will you need? In the end, your 
own education agency may cause the project to fall behind schedule. One state 
representative suggested that instead of emphasizing deadlines, agencies should 
put emphasis on quality results. 

> Ask for references from the vendor, perhaps even for the 

STAFF WITH WHOM YOU WILL BE WORKING 

Take the time to check these references and see what others thought of the 
services they received. Are the staff technically capable? Does the vendor have 
enough staff to support the endeavor? 

> Ask vendors to propose additional features that could 

IMPROVE THE SYSTEM 

Though you may not choose to accept their suggestions, they may have good 
ideas that could improve the functionality and utility of the system. 

> Get third-party opinions 

Have a variety of people beyond the writing team review the RFP to make sure it 
conveys your intent and is free of ambiguities. 

> Realize that your rep may not be perfect 

The requirements, as you have written them, may be flawed and could cause 
problems during implementation. Allow vendors to offer corrections or 
alternatives. 

On the safe side 

What might you include to protect your organization? In your RFP, consider including 
some safeguards and stipulations to ensure you get what you need from your vendor. Here 
are a few examples: 

Staffing 

> Maintain the right to change the vendor’s project manager if you are unhappy 
with his or her service. 

> Specify a limit to vendor staff turnover, perhaps to 25 percent or less. However, 
even if you say in the RFP that staff can only change by so much or in certain 
ways, know that turnover happens and you cannot always avoid it. If change must 
occur, ensure that the required knowledge transfer will be paid for by the vendor, 
not the taxpayers. 

> Specify that the project manager and staff you interview will actually work with 
you throughout the project. One education agency staffer suggested that vendors 
sometimes send their best team to the interview and then assign staff of lesser 
skill to the project once the deal is sealed. 

> Require that at least some vendor staff work onsite rather than remotely, if 
possible. Scheduling online or phone meetings can be a hassle and may impede 
regular and effective communication. If the vendor must work offsite, specify a 
number of onsite meetings and require that they cover travel expenses. Onsite 
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vendor staffing offers many advantages. For instance, both in-house staff and 
consultants are more apt to ask questions if the vendor is onsite. In addition, 
more meetings can be held and they can be more easily convened with specific 
individuals or groups of people. If vendor staff will be allowed to work remotely 
(an option that may be more cost-effective and provide more flexibility), devise 
an agreement to specify requirements such as the number of site meetings 
needed (and who will pay the expenses); communication planning such as virtual 
meetings, emailing, and instant messaging; and service-level agreements for 
off-site staff. 

Deliverables 

> Define deliverables in clear and unambiguous terms. Make sure these 
requirements have measurable results that hold the vendor accountable. 
Alternatively, ask the vendor to propose measures that can be used to assess the 
deliverables. 

> Require that deliverables meet certain quality criteria before they can be delivered 
(grammatically correct, spell-checked, formatted according to standards, etc.). 
Avoid wasting time correcting simple vendor mistakes. 

> Distinguish critical requirements from optional enhancements. In addition to 
the benefits discussed above, this can protect you by preventing the vendor 
from claiming that a requirement was an add-on that should cost extra. Clear 
specifications up front save time, frustration, and money later. 

Sustainability 

> Ask vendors to include ongoing maintenance and support in their proposals. 
Agencies report that many vendors just want to build the system, get paid, and 
then leave the in-house staff on their own. 

> Request or define a plan for knowledge transfer so that your staff is able to 
manage the system when the vendor leaves. 

Communication 

> If you intend to rely on your vendor to communicate with the education 
community, make sure they have an organized approach to communication 
or, alternatively, include in the proposal a communication plan that they must 
follow. 

• Determine who is going to write the communications: the vendor may know 
the product best and should have a communications staff— avoid having the 
project manager or a technology staffer write communications unless a piece 
is technical in nature. 

• Define who the audience(s) will be. 

• Define a communications schedule, or at what milestones or intervals 
communications will be sent out. 

• Define the ways communications will be made. 

• Figure out, in general, what the communications will say (see chapter 12). 

> Ask the vendor to provide sample communications. 

Take care when writing an RFP. Getting it right will save time and resources; and will 

help you get the system your stakeholders need. 
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Are We "There" Yet? 
Evaluating your LDS 




fa 

he best time to think about LDS evaluation is during the planning phase. A 
key question to consider early on, and throughout the development process, 
is “How will we know if we’ve been successful?” Planning ahead of time for the evaluation 
ensures you will know how to measure progress right from the beginning of development 
and implementation. 



During the planning phase, an agency should establish project goals and objectives 
that will become the very basis for the evaluation; as well as metrics on which the process 
will rely. In turn, your evaluator will be able to convert these objectives into evaluation 
designs and measurement protocols. Then, right from the start of the implementation, 
you (or the evaluator) can collect baseline data against which later data measures will be 
compared. 



How well does your LDS deliver in terms of your stated criteria for success? How 
does it enhance operations, improve data quality, and facilitate better data-driven 
decisionmaking? Your ability to evaluate the system depends largely on how you planned 
the project from the very beginning. How clear was your vision of the desired system? Did 
you identify unambiguous and measurable goals? Clarity at the beginning will help you 
and your stakeholders evaluate if your efforts are paying off, and help you make 
adjustments to refine the system over time (see chapters 8 and 14). 



The Subjectivity of "Success" 

Without clearly stated goals and criteria for measuring how close you come to reaching 
them, success may be depend on the user's vantage point. For instance, technologically 
savvy users may expect cutting edge applications, while average users may be more 
interested in usability and user-friendly interfaces (Miller 2000). The expectations set at 
the beginning of the project may also affect perceptions. Leaders of the marketing and 
outreach effort, for instance, will affect how others view the system. Care should therefore 
be taken not to create unrealistically high expectations. If you paint too ambitious a 
picture when trying to win support, stakeholders may end up dissatisfied with the results 
(Staples 2002). 
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Measuring LDS Success 

Before assessment begins, and ideally even before the project begins, agency leaders or 
stakeholder groups serving on an evaluation committee should reflect on some basic 
questions: 

> Who will evaluate the system? (Agencies commonly bring in an outside 
consultant.) 

> From whose perspective will the system’s success be judged (agency leaders, IT 
staff, teachers, etc.)? 

> What criteria will be used to judge success? (Identify well-defined assessment 
criteria aligned with the established LDS goals.) 

> Will greater emphasis be put on timeliness or on progress? 

> What types of information should be used to gauge success (for instance, 
qualitative or quantitative)? 

> What methods will be used to gather and analyze the information? 

And later, when interpreting evaluation results, consider changes that may have 
affected the project and the ability to achieve the original goals. Did political leadership 
change? Was there significant staff turnover in-house or in vendor staff? Were new laws 
passed? Did technology changes have any impact? Did new resources become available, or 
did a downturn in the economy affect the budget? 



Evaluation methods 

Evaluators may use various tools to gauge system success. Commonly used methods include 

> surveys (online and paper); 

> case studies; and 

> direct observation. 

Evaluation criteria and questions 

There are many possible questions you can use to evaluate your system's success. Table 5 
presents a collection of some evaluation criteria and questions. These should be answered 
using one or more of the above methodologies. While these questions are qualitative in 
nature, you should include solid, measurable criteria in the evaluation whenever possible. 
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Table 5. Sample evaluation questions 



Development and 
implementation 


■ How long did the project take? 

■ To what extent does the LDS match the original vision and design? 

■ To what extent has the implementation team included broad input into the system design? 

■ To what extent has the system been made available to a wide variety of stakeholders? 

■ To what extent has the system been "marketed" to those stakeholder groups? 


Cost and cost 
effectiveness 


■ What resources were used? 

■ What savings in time and money were realized? 


Measurable 

benefits/impact 


■ How have operations been made more efficient? 

■ To what extent has the system helped improve academic performance? 

■ To what extent has the system helped close achievement gaps? 

■ How has the system been used to target and enhance professional development? 

■ How has the system enhanced administrator decisionmaking? 

■ In what ways and to what extent has the system saved the enterprise money? 


Awareness and 
engagement 


■ How aware are stakeholders of the system? 

■ How well do stakeholders understand the system? 

■ What level of stakeholder engagement has been achieved? 


Use 


■ How widespread is the use of the system? 

■ Which system components are used and how are they applied? 

■ How widespread was training participation? 


Perceptions 


■ How well has the system met reporting and decisionmaking needs? 

■ How helpful are the reporting and analysis tools? 

■ Has the system made data adequately available and accessible? 

■ Are users getting the data they need? 

■ How effective has professional development been? 

■ What professional development initiatives have been created to improve LDS use? 


Lessons learned 


■ What are the barriers to effective use of the system? 

■ What improvements can be made to the system? 

■ How can professional development efforts be improved? 



Sources: IES 2007, Juillerat and James 2006, and Simon 2006. 
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Appendix A 



Overview of the LDS 
Guide Series 



The Traveling Through Time: The Forum Guide to Longitudinal Data Systems series consists of 
four guides intended to help state and local education agencies meet the many challenges 
involved in developing a robust longitudinal data system (LDS), populating them with 
quality data, and using this new information to improve the education system. 

Book One of Four: What is an LDS? focuses on the fundamental questions of what an 
LDS is (and what it is not), what steps should be taken to achieve a sound and successful 
system, what components make up an ideal system, and why such a system is of value 
to education. Book Two of Four: Planning and Developing an LDS, discusses the critical 
planning and development phases of an LDS project, from stakeholder engagement and 
needs assessment all the way through to system evaluation. Book Three of Four: Effectively 
Managing LDS Data will explore several fundamental challenges of data management, 
focusing on data governance, data quality, privacy, and security. Finally, Book Four of Four: 
Advanced LDS Usage, will address the effective utilization of LDS data, discussing the 
users and uses of the data; and emphasize the need for effective training and professional 
development. The figure on the next page lays out the major issues covered in each of the 
four books in this Fomm guide series. 
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Book l:What Is an LDS? 



i Understanding what an LDS is (and is not) 
i Appreciating the organizational steps needed to 
institute and effectively use an LDS 
i Identifying the technical features and 
capabilities of an effective LDS and the 
additional features that can enhance the 
system's utility 

i Recognizing the benefits of an LDS 



Book III: Effectively 
Managing LDS Data 

■ Defining governance structure 

■ Defining roles and responsibilities 

■ Collaborating to improve data quality and 
streamline operations 

■ Managing changes to the system 

■ Training staff to ensure data quality 

■ Auditing/validating data at all levels 

■ Establishing/following data standards 

■ Securing data to protect privacy 

■ Providing users access to key data 



Book II: Planning and 
Developing an LDS 

■ Engaging stakeholders 

■ Describing the current system 

■ Envisioning the desired system 

■ Defining needs, including data and functionality 

■ Gaining buy-in and funding 

■ Building relationships 

■ Writing an RFP 

■ Building or buying a system or components 

■ Transferring knowledge (e.g., from developers 
to staff) 

■ Defining and measuring success 

■ Refining the system 



Book IV: Advanced 
LDS Usage 

■ Collecting, storing, and delivering key data 

■ Developing useful reports to fulfill common 
data requests and needs 

■ Developing user-friendly data tools to facilitate 
access and analysis 

■ Training users to utilize the technology 

■ Building awareness, understanding, and 
analytical capacity 




Book One: What is an LDS? is available online at: http://nces.ed.gov/forum/pub_2010805.asp 



To order a print copy, please contact: 
ED Pubs 

U.S. Department of Education 
P.O. Box 22207 
Alexandria, VA 22304 



Or call toll free 1-877-4ED-PUBS or order online at http://www.edpubs.gov. 
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Appendix C 

Additional Resources 



Chapter 3. Project Management for an LDS Implementation 

Technology @ Your Fingertips 

National Forum on Education Statistics (1998). 
http://nces.ed.gov/pubs98/98293rev.pdf 

See sections 5.2 and 5.3 for some general lessons on project management of a technology 
solution. 

Governance Structure, LDS Project Team Organization, and 
Sustainability 

IES SLDS Grantee Meeting: Session V, SLDS (2006). 
http ://nces. ed.gov/programs/slds/gms_presentations. asp #2 

Representatives from several states discussed how their project teams were organized within 
their respective state education associations (SEA), and how decisions were made. 

South Carolina’s Governance Structure (Diagram) 

South Carolina Department of Education (2007). 

Available via LDS Share at http://nces.ed.gov/programs/slds/ldsshare/slds.aspx 
This diagram illustrates South Carolina’s LDS project governance structure. 

Chapter 4. Engaging Stakeholders: Bringing Everyone Along 

Alaska’s Initial Solicitation Letter Commissioner Signature 
(Invitation to Participate in Stakeholder Groups) 

Alaska Department of Education (2007). 

Available via LDS Share at http://nces.ed.gov/programs/slds/ldsshare/slds.aspx 
This item is a draft letter inviting stakeholder participation to Alaska’s Unity Project. 

Don’t Get Lost in Translation: The LDS and the Data Divas, 
Data Geeks, and Data Duffers 

South Carolina Department of Education (2007). 
http://nces. ed.gov/whatsnew/conferences/statsdc/2007/ppt/II_ C.zip 

This presentation from NCES’s 2007 Summer Data Conference reviews South Carolina’s 
approach to engaging stakeholders, getting the right people involved in the LDS project, 
and facilitating effective communication between these key players from various 
backgrounds to optimize results. 
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Stakeholder Involvement in Maryland 
Maryland Department of Education (2006). 
http://nces.ed.gov/programs/slds/zip/granteemeeting06_lb.zip 

This presentation given at the 2006 IES SLDS Grantee Meeting provides an overview of 
Maryland’s efforts to engage a wide variety of stakeholders. It includes the various types of 
stakeholders, their roles, and how needs assessment is conducted. Lessons learned and best 
practices are offered. Slide notes are also included. 

LDS Share 

The following documents are available at http://nces.ed.gov/programs/slds/ldsshare/slds.aspx: 

Alaska Unity Project: Functional Stakeholder 
Organization Chart 

Alaska Department of Education (2007). 

This item illustrates the Unity Project’s Functional Stakeholder Organization. 

External Communications Plan Example 

Michigan Department of Education (2007). 

This document represents an example and/or template external communications 
plan developed by the state of Michigan. 

Stakeholder Involvement in Designing and Developing 
Statewide Longitudinal Data Systems 
Wisconsin Department of Public Instmction (2006). 
http://nces.ed.gov/programs/slds/zip/granteemeeting06_la.zip 

This short presentation given at 2006 IES SLDS Grantee Meeting lists the challenges 
of engaging stakeholders. 

Chapter 5. Building State-District Relationships 

What Is the Role of the SEA in Providing LEAs With Access to 
Key Data for Instruction? 

Council of Chief State School Officers. Education Information Management Advisory 
Consortium LDS Taskforce (2007). 

What can the SEA do to help local agencies in the LDS effort? This brief covers the 
establishment of an identifier system, professional development, benchmarking, data 
management and business intelligence, and legislative support. 

Creating Longitudinal Data Systems: Lessons Learned by 
Leading States 

Data Quality Campaign (2006). 

http://www. dataquality campaign. org/resources/1 1 5 

This document summarizes findings and offers lessons learned from case studies of four 
leading states that vary in terms of LDS progress, public and political support for LDS, and 
the focus of their LDS efforts. 
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Chapter 6. Self-Assessment: You are Here, But... Where 
Exactly is That? 

Core Elements Completion for Establishing a Statewide 
Longitudinal Data System 

Institute of Education Sciences (2009). 

http ://nces. ed.gov/programs/slds/pdf/matrixofcoreelements.pdf 

This checklist can help your organization gauge where it is and where it needs to focus 
efforts in establishing an LDS. Agencies can reflect on which components they have, 
which they want or do not care to have, and the status of support and funding for those 
components. 

Forum Unified Education Technology Suite 

National Forum on Education Statistics (2005). 
http://nces.ed.gov/forum/pub_tech_suite.asp 

See chapter 2 for a discussion of needs-assessment and, to a lesser extent, self-assessment. 

2009-10 Survey Results 

Data Quality Campaign (DQC 2009). 

http://www.dataqualitycampaign.org/survey 

This is a sample survey of state education agencies, organized around the DQC’s 10 
essential elements and 7 state actions. It asks key questions about agency progress towards, 
and activities related to, these elements and actions. 

Chapter 7. Enterprise Architecture 

Enterprise Architecture 
Microsoft Developer Network. 

http://msdn.microsoft.com/en-us/architecture/hb469938.aspx 

This site includes links to a host of articles on EA including an introduction to Microsoft’s 
approach to EA and a discussion of the most popular EA frameworks. 

Enterprise Architecture as Strategy 
Ross, J.; Weill, P.; and Robertson, D. (2006). 

Boston, Massachusetts: Harvard Business School Press 

This book stmctures and explains the enterprise architecture (EA) approaches that top- 
performing private sector organizations have taken. 

Enterprise Architecture Frameworks 

Below is a list of some of the most popular EA frameworks. Microsoft notes that none of 
these EA approaches is sufficient for all situations and recommends using the most useful 
portions of each one to meet your organization’s needs. 
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Federal Enterprise Architecture (FEA) 

http ://www.whitehouse.gov/omb/e-gov/fea and http://www.whitehouse.gov/sites/default/ 
files /omb /assets /fea_docs /FEA _Practice_ Guidance _Nov_2 007. pdf 

The federal government has been a leader in developing and using EA and the 
private sector is following suit. 

The Open Group Architecture Framework and Architecture 
Development Method 

http ://www. opengroup.org/architecture/togaf8-doc/arch/toc. html 

The Open Group Architecture Framework (TOGAF) is a detailed method and 
supporting tools for developing an EA. It may be used freely by any organization for 
its own EA development. Architecture Development Method (ADM) is a “reliable, 
practical method... for defining business needs and developing an architecture that 
meets those needs, utilizing the elements of TOGAF and other architectural assets 
available to the organization.” In other words, ADM is the process for creating 
TOGAF. 

Zachman Framework 

Zachman International. 

http://zachmaninternational.com/index.php/the-zachman-framework 
One of the earliest EA frameworks, this is a logical structure for organizing the 
descriptive representations important to the management of an enterprise and the 
development of its systems. 

For more in-depth discussion of these and other frameworks, see Microsoft 
Corporation’s A Comparison of the Top Four Enterprise-Architecture Methodologies at 
http://msdn.microsoft.com/en-us/architecture/bb466232.aspx. 



Chapter 8. Needs-Assessment: Defining "There" 

Technology @ Your Fingertips 

National Forum on Education Statistics (1998). 
http://nces.ed.gov/pubs98/98293rev.pdf 

Chapter 2 contains a helpful discussion of needs assessment. While not specific to an LDS 
project, it offers some relevant guidelines for finding out what functional and technological 
needs your stakeholders have. It also offers tips on creating a statement of needs. 

Creating a Statewide Educational Data System for 
Accountability and Improvement: A Comprehensive Information 
and Assessment System for Making Evidence-Based Change at 
School-, District-, and Policy-Levels. 

Felner, R.; Bolton, N.; Seitsinger, A.; Brand, S.; and Burns, A. (2008). Psychology in the 
Schools 45(3), 235-256. 

This article provides a good example of how designers can think big from the outset, 
defining their needs and developing a system guided by more ambitious goals than 
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usual. It covers many areas of the education system that may be measured, including 
developmental, educational, fiscal, and policy conditions, as well as students' academic 
performance and developmental and educational needs. 

Effective Use of Electronic Data Systems: A Readiness Guide for 
School and District Leaders 

Learning Point Associates and the Educational Service Agency (ESA) Alliance of the 
Midwest (2006). 

http://www.learningpt.org/pdfs/datause/DataReadinessTool.pdf 

This guide is intended to help districts figure out what they want from a data system, 
identify appropriate questions to ask about a system, and assess the district’s readiness and 
capacity to use one effectively. 

Beyond Test Scores: Leading Indicators for Education 

Annenberg Institute for School Reform (2008). 
http://www.annenberginstitute.org/pdf/leadingindicators.pdf 

This study looks at leading indicators used to identify signs of academic progress before 
test scores come in. These indicators may be useful in helping agencies think about the 
questions they want to explore and the data needed to answer them. 

LDS Share 

The following documents are available at http://nces.ed.gov/programs/slds/ldsshare/slds.aspx: 

Longitudinal Data Systems: Summary of Current and 

Potential Issues 

Maryland Department of Education (2006). 

This document discusses current issues and potential uses of LDSs. It reviews the 
basic requirements of an LDS as well as the options that can be built into the 
system. It highlights themes and poses questions that may be used to inform the 
design of a needs-assessment process. 

Maryland Longitudinal Data System Needs Assessment 
Guidelines for Internal Stakeholders 

Maryland Department of Education (2006). 

This document represents materials that Maryland’s project staff use to conduct a 
needs assessment for internal stakeholders. The document also contains proposed 
topics for needs assessment for external stakeholders. 

Questionnaire for Teacher Specialists 

South Carolina Department of Education (2007). 

This questionnaire lists a host of questions the state’s LDS can answer. It was given 
to teacher specialists who were asked to rate the questions in terms of the value their 
answers would offer. 

SLED Focus Group Requirements Traceability Matrix 
District of Columbia (2007). 

This file summarizes the findings of numerous focus groups the District of 
Columbia held with a variety of stakeholders: the mayor’s office, community/ 
principal groups, and functional groups (e.g., special education, charter schools, 
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funders, researchers). Potential data requirements for a state longitudinal data 
warehouse (SLED) were identified and rated on a scale of one to three by all 
stakeholder groups. The file also includes the focus group schedule. 

Chapter 9. Data: Knowing What You Have, Identifying What 
You Need 

NCES Handbooks Online 

National Center for Education Statistics. 
http ://nces. ed.gov/progmms/handbook 

These handbooks provide guidance on consistency in data definitions and maintenance 
for education data, so that such data can be accurately aggregated and analyzed. Use 
this searchable web tool to find standard data elements for students, staff, and education 
institutions; along with standard definitions and recommended values or responses for 
each element. 

National Education Data Model 

U.S. Department of Education. 

http ://nces. ed.gov/forum/datamodel 

The National Education Data Model catalogs the data used in P-12 education and 
describes the relationships among those data. It is designed to be used as a reference tool 
to facilitate the identification, merging, and matching of data across different systems; to 
provide similar descriptions across local education agency (LEA) systems, across LEAs, and 
from LEAs to the state and federal government; and to specify the content and stmcture of 
logical and physical data models. 

Common Data Elements for Education Technology Assessment 

State Educational Technology Directors Association. 
http://www. setda.org/toolkit/nlitoolkit/cde/cde06. htm 

This toolkit presents the common data elements SETDA has identified for tracking state 
progress towards the goals of the technology section of the No Child Left Behind Act of 
2001 (NCLB). 

Every School Day Counts: The Forum Guide to Collecting and 
Using Attendance Data 

National Forum on Education Statistics (2009). 
http://nces.ed.gov/forum/pub_2009804.asp 

This guide defines “attending/present” and “not attending/ absent,” categorizes attendance 
codes in an exhaustive and mutually exclusive way, and supports improved attendance data 
quality and comparability between states and districts. 

Accounting for Every Student: A Taxonomy for Standard 
Student Exit Codes 

National Forum on Education Statistics (2006). 
http://nces.ed.gov/forum/pub_2006804.asp 

This guidebook presents “best practice” advice for tracking and maintaining information 
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on enrollment status. It presents an exhaustive and mutually exclusive taxonomy of exit 
codes. 

Longitudinal Data Systems: Summary of Current and 
Potential Issues 

Maryland Department of Education (2006). 

Available via LDS Share at http://nces.ed.gov/programs/slds/ldsshare/slds.aspx 
This document summarizes information related to longitudinal data systems in education, 
exploring current issues and potential uses. It will guide the external stakeholder needs 
assessment process, highlighting themes and posing questions to be addressed in 
interviews, surveys, and focus groups. 

Getting the Evidence for Evidence-Based Initiatives: How the 
Midwest States Use Data Systems to Improve Education Processes 
and Outcomes 

National Center for Education Evaluation and Regional Assistance (2007). Issues and 
Answers Report (REL 2007-016). U.S. Department of Education, Institute of Education 
Sciences. 

http://ies.ed.gov/ncee/edlabs/regions/midwest/pdf/rel_2007016_sum.pdf 
This report reviews the progress of several midwestern states in developing LDSs and the 
use of data systems in general. Based on interviews with SEA officials and federal agency 
staff, the authors review the work that was done, the challenges that were faced, and the 
current requirements being pursued by the states. 

Illinois State Board of Education SIS Data Elements, Approved 
Codes and Indicators 
Illinois State Board of Education. 

http ://www. isbe. net/sis/html/data_elements. htm 

This page contains documentation for all data elements collected through the Illinois State 
Board of Education’s (ISBE) student information system (SIS), including demographics, 
enrollment, assessment, English language learners, early childhood education, discipline, 
student identifiers, and more. 

Creating a Comprehensive Teacher Data System 

Center for Strengthening the Teaching Profession. 
http://www.dataqualitycampaign.org/resources/94 

Page 6 of this report includes a list of data elements suggested for a comprehensive staff 
data system. 

Forum Guide to Core Finance Data Elements 

National Forum on Education Statistics (2007). 

http://nces.ed.gov/forum/pub_2007801.asp 

This document provides an overview of key finance data terms. 

Linking Spending and Student Achievement: Managing Inputs, 
Processes, and Outcomes 
Data Quality Campaign (DQC 2008). 

http://dataqualitycampaign.org/files/meetings-dqc_quarterly_issue_brief042408.pdf 
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This issue brief explores the benefits of linking financial and student achievement data. It 
discusses the challenges of improving current financial systems, and includes two case studies. 

Facilities Information Management: A Guide for State and Local 
Education Agencies 

National Forum on Education Statistics (2003). 
http://nces.ed.gov/forum/pub_2003400.asp 

This guide provides a framework for identifying a basic set of school facilities data 
elements and definitions that will meet the information needs of school and community 
decisionmakers, school facility managers, and the general public. 

Linking Teacher and Student Data to Improve Teacher and 
Teaching Quality 

Data Quality Campaign (DQC 2007). 

http : //dataquality campaign . o rg /files/ Meetings- DQC _Quarterly _Issue_B rief_03 120 7 . pdf 

This issue brief discusses information about teachers that may be tracked in an LDS, and 
the benefits those data can offer when linked to student data. 

Victoria Bernhardt Web Page 

http://www.eyeoneducation.com/authors/bernhardt_v.asp 

This author has written a number of publications discussing the types of data that may be 
useful to educators at the local level. This site links to many of these resources. 

Ruth Johnson Web Page 

http://www.sagepub.com/authorDetails.nav?contribId=520833 

This author has written a number of publications discussing the types of data that may be 
useful to educators at the local level. This site links to a many of these resources. 

Chapter 10. Some Critical "Abilities": Interoperability and 
Portability 

The Right Data to the Right People at the Right Time: How 
Interoperable Data Help America’s Students Succeed 
Data Quality Campaign (DQC 2007). 

http://dataqualitycampaign.org/files/Meetings-DQC_Quarterly_Issue_Brief_061307.pdf 

This paper reviews the needs for, benefits of, and concurrent efforts to establish 
interoperable education systems. It offers several key definitions, a case study section, and 
a list of interoperability examples from other industries. 

SIF Association Implementation Toolkit 

SIF Association (2009). 
http://www.sifinfo.org/us/tool_kit.asp 

This collection of documents are intended to help education institutions with the SIF 
implementation process. It includes planning questions (scope, desired automation, data 
needs, expected changes), RFP language, an implementation planning toolkit, and SIF 
support resources. 
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Analysis of Costs and Benefits Associated with Implementing SIF 

SIF Association (2006). 

http ://www. sifinfo. org/upload/Docs/SIF _Cost_Benefit_Analy sis _Summary_June20 _Final.pdf 
This third-party study looks at three school districts’ experiences with SIF implementation 
and concludes that SIF standards helped to dramatically improve data interoperability, 
student achievement, funding increases, and student services. This study may be used to 
bolster the case for SIF implementation. 

Data Interoperability in PK-12 School Applications 

SIF Association (2006). 

http://www.sifinfo.org/upload/press/532964_Interoperability%20and%20pkl2.pdf 

This brief discusses interoperability and SIF, its benefits to P-12 education, current 
progress, and suggestions for moving forward. 

SIFA University 

SIF Association (2006). 

http ://www. sifinfo. org/us/sifau.asp 

This site offers online overview modules and information on training courses offered by 
the SIF Association. 

SCORM and SIF— Leveraging Work for Successful Solutions 

SIF Association (2006). 

http://www.sifinfo.org/upload/press/289bza_scorm and sif_final.pdf 

This brief discusses two organizations’ standards for interoperability: SIFA’s standards 
(SIF) and those of Advanced Distributed Learning (ADL), which are called SCORM. 

This piece introduces each and suggests that organizations consider both when pursuing 
interoperability. 

SIF Specifications 

SIF Association. 

http://www.sifinfo.org/us/sif-specification.asp 

This page includes the latest SIF implementation specifications. Access is free. 

Data Portability Project 

http://www. dataportability. org 

The Data Portability Project promotes the idea that individuals have control over their data 
(regardless of the entity that holds the data) and can determine how the information can be 
use and by whom. 



In the news... 

The SIFication of America 

Schater, R. (2009). District Administration, March 2009. 
http://www.districtadministration.com/viewarticle.aspx?articleid=1970 

This article reports on districts' growing effort to streamline data exchange and 
management through the adoption of SIF. 
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Chapter 1 1 . Staying "There": Ensuring System Sustainability 

Wisconsin Case Study: Building a Student-Level Longitudinal 
Data System 

Data Quality Campaign (DQC 2006). 

http://www.dataqualitycampaign.org/files/State_Specific-Wisconsin_2006_Site_yisit.pdf 

Pages 4-5 of this case study include a list of estimated costs to the state to implement a 
vendor-developed student ID system and student-level enrollment data collection. Costs 
are broken up by year and source. A discussion of costs and extra burdens imposed on 
districts is also provided. 

South Carolina Case Study: Building a Student-Level 
Longitudinal Data System 

Data Quality Campaign (DQC 2007). 

http://www.dataqualitycampaign.org/files/Publications-South_Carolina_Case_Study_ 

Building_a_Student-level_Longitudinal_Data_System-ll0207.pdf 

Pages 6-7 of this case study include a list of estimated costs to the state to implement SIF 
standards, bring in extra staff, upgrade infrastructure, and implement a state data manager 
and a new data warehouse. A discussion of costs and extra burdens imposed on districts is 
also provided. 

State Data to Improve Achievement 

Washington Office of the Superintendent of Public Instruction (2006). 
http://www.dataqualitycampaign.org/resources/317 

This document is a request to the Washington Legislature to fully fund a two-year, $2.9 
million statewide longitudinal student database project. The document sets the stage with 
relevant history; and discusses the need for the proposed system, as well as the associated 
costs. The system is meant to facilitate daily data extractions from districts, replace existing 
single-year achievement databases with longitudinal achievement and demographic ones, 
and build and implement value-add tools. 

Chapter 12. Marketing and Communicating About Your LDS 

Creating Longitudinal Data Systems: Lessons Learned by 

Leading States 

Data Quality Campaign (DQC 2006). 

http://www. dataquality campaign. org/resources/1 1 5 

This document summarizes findings and offers lessons learned from case studies of four 
leading states that vary in terms of LDS progress, public and political support for LDS, and 
the focus of their LDS efforts. 

Strategic Communications Planning 
Fleet, D. (2008). DaveFleet.com. 

http ://davefleet. com/2008 /08/strategic-communications-planning-afree-ebook 

Written from a business perspective but intended for a broader audience including the 

public sector, this free, electronic book guides the reader through the development of 
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a communications plan. Covered marketing goals relevant to an LDS project include 
educating stakeholders, building support, and creating demand. This resource may be 
useful from the early stages of stakeholder analysis through crafting and spreading your 
message, estimating costs, and evaluating the results of your efforts. 

Marketing Your Field of Dreams: The Process of Obtaining and 
Sustaining Buy-in 
ESP Solutions (2007). 

http ://www. espsolutionsgroup.com/espweb/assets/files/ESP_Marketing_Field_D reams_ORG.pdf 

Written from the state’s perspective, this document offers strategies on how to use various 
marketing techniques to gain buy-in for a technology project. A marketing plan is outlined 
to help your organization gain stakeholder support. 

Education Leadership Toolkit: The Communication Plan 

National School Boards Foundation. 
http ://‘www. nsba.org/sbot/toolkit/tcp. html 

This free, online resource addresses issues around technology and education and offers 
many tips, articles, case studies and other resources for education leaders. Though it is 
designed for a district-level audience, much of the advice may be used more broadly. 

This section focuses on developing a successful communication plan. See the community 
involvement section as well: http://www.nsba.org/sbot/toolkit/ComInv.html. 

DATA/KIDS III Projects: Integrated Communications Plan 

Oregon Department of Education (2009). 
http://www.oregondataproject.org/content/communications-plan 

This document details the state’s communication plan and governance structure for its 
professional development program and data system. 

Developing Political Support and the Will to Build and Use 
Longitudinal Education Data Systems 

Pennsylvania Department of Education and Florida Department of Education (2006) 
http://nces.ed.gov/whatsnew/conferences/statsdc2006/conference/presents/iv_d.zip 

This presentation reviews lessons learned by Pennsylvania and Florida when marketing 
their states’ LDSs. Emphasis is on having and consistently communicating a clear vision, 
and on being persistent to maintain support over the long term. 

External Communications Plan Example 

Michigan Department of Education (2007). 

Available via LDS Share at http://nces.ed.gov/programs/slds/ldsshare/slds.aspx 

This document presents a template for organizing committees and facilitating ongoing 
communications about an LDS project. It outlines the various roles and activities of these 
groups and their members, as well as suggesting ground rules for meetings. 

Marketing and Communicating the LDS Project— Florida 

Florida Department of Education. 

http://nces.ed.gov/programs/slds/zip/granteemeeting06_10a.zip 
This short presentation offers some tips on marketing an LDS. 
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Chapter 13. Procurement Planning: Build or Buy? 

An Examination of the T rad e-off Between Internal and External 
IT Capabilities 

Nevo, S.; Wade, M.; and Cook., W. (2007). Journal of Strategic Information Systems, 16(1); 5-23. 
This study explores the tension between internal and external IT capabilities on the 
realization of enhanced IT productivity. It focuses on short-term, small scale consulting 
rather than the large scale, multiyear outsourcing that often occurs in LDS development. In 
that respect, its applicability is limited here. However, it offers insights into the relationship 
between in-house staff and vendors; and may help enlighten your decision on whether to 
buy or build, and how to proceed down the path you choose. 

Data Quality Campaign Vendors Page 

http://www.dataqualitycampaign.org/survey/vendors 

This page lists which states are using vendors to help them develop LDS components. 
These results are from the 2008 DQC Survey. 

Technology @ Your Fingertips 

National Forum on Education Statistics (1998). 
http://nces.ed.gov/forum/pub_98293.asp 

See pages 50-57 for general discussions on conducting a build vs. buy analysis, finding a 
product to fit your needs, and other related issues. While not specific to an LDS project, 
this resource offers some relevant tips. Page 57 includes a list of sample questions that 
might be used to interview vendor references. 

Data Tools for School Improvement 

Bernhardt, V. (2005). Educational Leadership 62(5), 66-69. 
http://eff.csuchico.edu/downloads/datatools.pdf 

This article discusses the three types of data tools available— data warehouses, student 
information systems, and instmctional management systems— and their uses; as well as how 
to make good purchasing decisions, get what your organization wants and needs, how to 
deal with vendors, etc. 

Chapter 14. Writing a Strong Request for Proposals 

Lessons Learned: Writing RFPs for State Data Systems 

Smith, N. (2004). Education Commission of the States/InfoSynthesis. 
http://www.dataqualitycampaign.org/resources/112 

This report summarizes findings from a survey of three state education agencies and offers 
a host of RFP writing principles and tips. Two RFP examples are provided at the end of the 
document. 
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Writing RFPs for State Data Systems: Lessons Learned 

Washington Office of Superintendent of Public Instruction, Michigan Center for 

Educational Performance and Information, and Ohio Department of Education (2007). 

http://nces.ed.gov/whatsnew/conferences/mis/2007/presentations/vi_c.zip 

This file includes presentations from three states on lessons learned in RFP writing. 

LDS Share 

Institute of Education Sciences (IES), U.S. Department of Education. 
http ://nces. ed.gov/programs/slds/ldsshare/slds. aspx 

Coordinated by the IES Statewide Longitudinal Data Systems Grant Program, this web 
resource contains a collection of state-level RFPs submitted by education agencies. 

SIF Association Recommended RFP Language 

SIF Association (2009). 

http://www.sifinfo.org/us/upload/implement/BF3E8E_SIF%20RFP%20Language.pdf 

This document offers language to include in your RFP if your agency wants to purchase 
an application based on SIF standards. It requests documentation of SIF certification, SIF 
implementation experience, agent costs, zone integration server(s) offered, SIF Association 
participation, and SIF support offered to clients. 

Virginia Case Study: Building a Student-Level Longitudinal Data 
System 

Data Quality Campaign (DQC 2006). 

http://www.dataqualitycampaign.org/files/State_Specific-Virginia_2006_Site_Yisit.pdf 
See pages 6-7 for helpful tips on RFP writing and evaluating bids. 

Forum Guide to Decision Support Systems: A Resource for 
Educators 

National Forum on Education Statistics (2006). 
http : //nces. ed.gov/forum /pub_2006807.asp 

See this guide’s appendix for an overview of issues that may be included in a decision 
support system RFP. There is considerable overlap between these requirements and those 
that might be included in an LDS. 

Technology @ Your Fingertips 

National Forum on Education Statistics (1998). 
http://nces.ed.gov/puhs98/98293rev.pdf 

Pages 57-59 of this resource offer a brief introduction of RFPs and a sample RFP table of 
contents to help guide the writing process. It also discusses requests for information (RFI) 
and interviewing references, including questions to ask interviewees. 
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Chapter 1 5. Are We "There" Yet? Evaluating Your LDS 

Evaluation of the Effectiveness of Ohio’s D3A2 Initiative 

Juillerat, B. and James, E. (2006). Ohio Department of Education. 
http://nces.ed.gov/programs/slds/zip/gmnteemeeting06_4a.zip 

This presentation introduces Ohio’s LDS evaluation plans and summarizes the state’s 
system as well as the criteria for, and methods used in, a planned system evaluation. 

Arkansas Department of Education LDS Evaluation 

Metis Associates. IES SLDS Grantee Meeting Presentation (2006). 
http://nces.ed.gov/programs/slds/zip/granteemeeting06_4b.zip 

This presentation by Arkansas’ evaluation vendor gives an overview of the state’s LDS and 
discusses its approach to evaluating that system. A basic overview of evaluation criteria, 
methods, and timeline are provided. 

Everybody Loves Evaluation 

Indiana Department of Education (2008). 
http://nces.ed.gov/programs/slds/zip/08_F_08a.zip 

This presentation details the state’s evaluation process. It outlines the purposes of the LDS 
evaluation, procedures used, activities, and sample findings. 

You Say You Want Evaluation 

Arkansas Department of Education (2008). 
http://nces.ed.gov/programs/slds/zip/08_F_08b.zip 

This presentation reviews the findings of a survey of districts intended to evaluate 
perceptions of the state’s LDS. Questions seek to ascertain users’ perceptions about the 
system’s most useful functions, ease and frequency of use, success in improving data 
quality, and effects on student performance. 

LDS Share 

The following evaluation-related documents are available at 
http://nces. ed.gov/programs/slds/ldsshare/slds. aspx: 

Longitudinal Data System Implementation and Impact 
Evaluation RFP 

Ohio Department of Education (2006). 

Ohio solicited competitive proposals for its LDS Implementation and Impact 
Evaluation of 2006-2009, and this RFP is the result of that request. The purpose 
of the evaluation is to measure the implementation and impact of the LDS, 
including related professional development, quality, and effectiveness in meeting the 
reporting and decision support needs of all its key stakeholders; and, ultimately, its 
effectiveness in closing achievement gaps and in generating academic improvement 
in all students. 
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Supporting Ohio’s Longitudinal Data System through 
Evaluation: A Proposal to the Ohio Department of Education 

Ohio Department of Education (2006). 

This document is the winning proposal, submitted by Hezel Associates LLC, to 
Ohio’s LDS Implementation and Impact Evaluation RFP. 

Supporting D3A2 Professional Development through 
Evaluation: A Report to the Ohio Department of Education 

Ohio Department of Education (2007). 

This report offers key findings from an evaluation of Ohio’s LDS, focusing on 
data usage practices and professional development efforts. Key findings and 
recommendations are followed by discussions of methodology and more detailed 
discussions of evaluation findings. 

Supporting D3A2 Professional Development through 
Evaluation: A Preliminary Annual Report to the Ohio 
Department of Education 

Ohio Department of Education (2007). 

This LDS evaluation report follows up on the previous one. It offers key 
recommendations for future LDS efforts, focusing on implementation and 
expanding on earlier recommendations to improve professional development efforts. 
Key findings and recommendations are followed by discussions of methodology and 
more detailed discussions of evaluation findings. 
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Appendix D 



INFORMATION TECHNOLOGY 

Audits 



Considering the large outlay of resources involved in LDS development, many states will 
want to review how their tax dollars are being spent. As a result, your agency may become 
involved in federal or state auditing processes and procedures. Agencies developing LDSs 
may therefore be interested in the available types of information technology audits and 
what they encompass. 

According to Wikipedia, an information technology audit, or information 
systems audit, is “an examination of the controls within an information technology (IT) 
infrastmcture. An IT audit is the process of collecting and evaluating evidence of an 
organization’s information systems, practices, and operations. The evaluation of obtained 
evidence determines if the information systems are safeguarding assets, maintaining data 
integrity, and operating effectively and efficiently to achieve the organization’s goals or 
objectives. These reviews may be performed in conjunction with a financial statement 
audit, internal audit, or other form of attestation engagement.” (http://en.wikipedia.org) 

If your agency can afford it, the creation of an internal audit team can be very 
beneficial, potentially saving your organization time and money. The first step in 
developing an audit team is to define the information technology audit functions. A 
useful approach is to define your IT function into layers: IT management, technical 
infrastmcture, applications, and external connections (Juergens 2006). 

As Juergens states, IT management “comprises the set of people, policies, 
procedures, and processes that manage the IT environment.” This layer also includes 
system monitoring, programming, planning, management of outsourced vendors, and 
IT governance. Technical infrastmcture usually refers to systems that underlie, support, 
and enable the primary business applications such as operating systems, databases, and 
networks. Applications may be classified into two categories: transactional and support. 
Transactional applications consist primarily of software that processes and records business 
transactions. Support applications are specialized software programs that facilitate business 
activities but generally do not process transactions such as email programs, fax software, 
and design software. External connections are those that your network connects to, such as 
the Internet or other business networks. 

Organizations developing LDSs may be interested in two types of controls to be 
audited: application controls and information technology general controls. 

Application controls are specific to each application and relate to the transactions and 
data pertaining to each computer-based application system. The objectives of application 
controls are to ensure the completeness and accuracy of records and the validity of the 
entries made resulting from programmed processing activities. Examples of application 
controls include: 



Book Two of Four: Planning and Developing an LDS 



109 




> Input controls: These controls are used mainly to check the integrity of data 
entered into a business application, whether the data is entered directly by staff, 
remotely by a business partner, or through a web-enabled application or interface. 
Data input is checked to ensure it remains within specified parameters. 

> Processing controls: These controls provide an automated means to ensure 
processing is complete, accurate, and authorized. 

> Output controls: These controls address what is done with the data, and should 
compare actual results with the intended result by checking the output against 
the input. 

> Integrity controls: These controls monitor data being processed and in storage to 
ensure they remain consistent and correct. 

> Management trail: Processing history controls, often referred to as an audit 
trail, enables management to identify the transactions and events they record by 
tracking transactions from their source to their output, and by tracing backward. 
These controls also monitor the effectiveness of other controls, and identify errors 
as close to their sources as possible. (Bellino and Hunt 2007) 

General controls apply to all system components, processes, and data present in an 
organization or systems environment. The objectives of these controls are to ensure the 
appropriate development and implementation of applications; as well as the integrity of 
program and data files, and of computer operations. The most common general controls are 

> logical access controls over infrastructure, applications, and data; 

> system development life-cycle controls; 

> program change management controls; 

> physical security controls over the data center; 

> system and data backup and recovery controls; and 

> computer operation controls. 

(Bellino and Hunt 2007) 



Risk Assessment 

One of the many benefits of an information audits is to identify risks to your organization. 
Juergens (2006) has identified the following as possible risks: 

> availability, when the system is unavailable for use; 

> security, when unauthorized access to systems occurs; 

> integrity, when the data is incomplete or inaccurate; 

> confidentiality, when information is not kept secret; 

> effectiveness, when the system does not deliver an intended or expected function; 
and 

> efficiency, when the system causes a sub-optimal use of resources. 
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The Audi! Process 

The following is an example provided by Microsoft Corporation’s Regulatory Compliance 
Planning Guide (Microsoft 2006) on what to expect during an IT audit. 

1. Plan the audit (auditor). 

2. Hold audit kickoff meeting (auditor/organization). 

3. Gather data and test IT controls (auditor/organization). 

4. Remediate identified deficiencies (organization). 

5. Test improved controls (auditor/organization). 

6. Analyze and report findings (auditor). 

7. Respond to findings (organization). 

8. Issue final report (auditor). 

IT audits create challenges for organizations by forcing them to come into, and 
maintain, compliance. But audits also offer benefits, increasing your organization’s process 
improvement and business advantage. 
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Appendix E 

Forum and Other NCES 
Resources 

The Forum Guide to Data Ethics (NFES 2010-801) 

http://nces. ed.gov/forum/pub_2 01 0801. asp 

Every day, educators collect and use data about students, staff, and 
schools. Some of these data originate in individual student and staff 
records that are confidential or otherwise sensitive. And even those 
data that are a matter of public record, such as aggregate school 
enrollment, need to be accessed, presented, and used in an ethically 
responsible manner. While laws set the legal parameters that govern 
data use, ethics establish fundamental principles of “right and 
wrong” that are critical to the appropriate management and use of education data in the 
technology age. This guide reflects the experience and judgment of experienced data 
managers; while there is no mandate to follow these principles, the authors hope that the 
contents will prove a useful reference to others in their work. 

Crisis Data Management: A Forum Guide to Collecting and 
Managing Data about Displaced Students (NFES 2010-804) 

B http://nces. ed.gov/forum/pub_201 0804. asp 

This publication provides guidelines that can be used by elementary 
and secondary education agencies to establish policies and procedures 
for collecting and managing education data before, during, and after a 
crisis. 




The Forum Guide to Metadata: The Meaning Behind Education 
Data (NFES 2009-805) 

http://nces.ed.gov/forum/pub_2009805.asp 
This guide offers best practice concepts, definitions, 
implementation strategies, and templates/ tools for an audience of 
data, technology, and program staff in state and local education 
agencies. This resource was developed to improve these audiences’ 
awareness and understanding of metadata and, subsequently, the 
quality of the data in the systems they maintain. 




METADATA 
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Every School Day Counts: Forum Guide to Collecting and 
Using Attendance Data (NFES 2009-804) 

http://nces. ed.gov/forum/pub_2 009804. asp 

This guide offers best practice suggestions on collecting and using 
student attendance data to improve performance. It includes a 
standard set of codes to make attendance data comparable across 
districts and states. The publication includes real-life examples of how 
attendance information has been used by school districts. 




iK a n t ftooks Ohline — 



NCES Handbooks and NCES Handbooks Online 

http://www.nces.ed.gov/programs/handbook 
The NCES Handbooks are a valuable source of metadata 
for organizations and individuals interested in education data. These print and online 
resources define standard education terms for students, staff, schools, local education 
agencies (LEA), intermediate education agencies, and state education agencies (SEA). The 
Handbooks are intended as reference documents for public and private organizations, 
including education institutions and early childhood centers; as well as education 
researchers and other users of education data. In order to improve access to this valuable 
resource, NCES has also developed the NCES Handbooks Online, a web-based tool that 
allows users to view and download information via an electronic table of contents, a drill- 
down finder, element name and first letter searches, and advanced query options. 



National Education Data Model 

m http ://nces. ed.gov/forum/datamodel/index. aspx 

The National Education Data Model (NEDM) is the first 
non-proprietary, national education data model developed to help schools, LEAs, and 
states design or guide the selection of systems for instructional delivery, data-driven 
decisionmaking, data collection, operations, and reporting. The model provides a national 
blueprint to help schools evaluate and improve instructional tools; communicate needs to 
their umbrella agency or to vendors; enhance the movement of student information from 
one LEA to another; and, in the end, have better tools to inform instruction. NEDM can 
be used by educators, vendors, and researchers to understand the information required for 
teaching, learning, and administrative systems. 



Managing an Identity Crisis: Forum Guide to Implementing New 
Federal Race and Ethnicity Categories (NFES 2008-802) 

http://nces. ed.gov /forum /pub _2008802. asp 
This best-practice guide was developed to help state and local 
education agencies implement the new federal race and ethnicity 
categories, thereby reducing redundant efforts within and across states, 
improving data comparability, and minimizing reporting burden. 

Users may select and adopt strategies that will help them quickly begin 
the process of implementation in their agencies. 
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Forum Guide to Core Finance Data Elements (NFES 2007-801) 

http://nces.ed.gov/formn/pub_2007801.asp 
This publication establishes current and consistent terms and 
definitions for maintaining, collecting, reporting, and exchanging 
comparable information related to education finances. It is designed 
to accompany Financial Accounting for Local and State School 
Systems: 2003 Edition by identifying common reporting requirements 
and defining frequently used indicators and calculations that use data 
elements from accounting and other data systems. 

Forum Curriculum for Improving Education Data: A Resource for 
Local Education Agencies (NFES 2007-808) 

http://nces. ed.gov/forum/pub_2 007808. asp 

This resource supports efforts to improve the quality of education 
data by serving as training materials for K-12 school and district staff. 
It provides lesson plans, instructional handouts, and other resources; 
and presents concepts necessary to help schools develop a culture for 
improving data quality. 





Forum Guide to Decision Support Systems: A Resource for 
Educators (NFES 2006-807) 




http : //nces. ed.gov/fo rum/pu b_2006807.asp 

This Forum guide was developed to help the education community 
better understand what decision support systems are, how they are 
configured, how they operate, and how they might be developed and 
implemented in an education setting. 



Forum Guide to Virtual Education (NFES 2006-803) 

■ http://nces.ed.gov/forum/pub_2006803.asp 

This publication offers recommendations for collecting accurate, 
comparable, and useful data about virtual education in elementary and 
secondary education settings. It highlights policy questions and data 
elements critical to meeting the information needs of policymakers, 
administrators, instmctors, and parents of students involved in virtual 
education. 
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Forum Guide to the Privacy of Student Information: 
A Resource for Schools (NFES 2006-805) 




and “directory information,” and offers guidance for developing 
appropriate privacy policies and information disclosure procedures 
related to military recmiting, parental rights and annual notification, videotaping, online 
information, media releases, surveillance cameras, and health-related information. 



http://nces.ed.gov/forum/pub_2006805.asp 

This publication was written to help school and local education agency 
staff better understand and apply the Family Educational Rights and 
Privacy Act (FERPA), a federal law that protects the privacy interests 
of parents and students with respect to information maintained in 
student education records. It defines terms such as “education records” 



Accounting for Every Student: A Taxonomy for Standard Student 
Exit Codes (NFES 2006-804) 

http://nces. ed.gov/forum/pub_2 006804. asp 

This publication was developed to help education agencies develop 
effective information systems for tracking students’ enrollment status. 
It presents a student-level exit code taxonomy for states and districts 
that accounts for 100 percent (not 90 or 110 percent) of all students. It 
also offers “best practice” advice regarding tracking students, collecting 
exit codes data, and distinguishing among high school completion 
credentials. 




Forum Guide to Education Indicators (NFES 2005-802) 

http://nces.ed.gov/forum/pub_2005802.asp 

This publication provides encyclopedia-type entries for 44 commonly 
used education indicators. Each indicator entry includes a definition, 
recommended uses, caveats and cautions, related policy questions, data 
element components, a formula, commonly reported subgroups, and 
display suggestions. The document will help readers better understand 
how to appropriately develop, apply, and interpret commonly used 
education indicators. 



Forum Guide to Building a Culture of Quality Data 
(NFES 2005-801) 

http://nces.ed.gov/forum/pub_2005801.asp 

This publication focuses on data entry: getting things done right at the 
source. It recommends a practical process for developing a “Culture 
of Quality Data” based around individual tip sheets for individuals 
involved in providing data, including principals, teachers, office staff, 
school board members, superintendents, data stewards, and technology 
staff. 
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Forum Unified Ed ucation Technology Suite (2005) 

http://nces.ed.gov/forum/pub_tech_suite.asp 
This publication presents a practical, comprehensive, and tested 
approach to assessing, acquiring, instituting, managing, securing, 
and using technology in education settings. It is written for 
individuals without extensive experience with technology who have been tasked with 
leading technology initiatives in a school or district setting,. 



Forum Guide to Protecting the Privacy of Student Information: 
State and Local Education Agencies (NCES 2004-330) 




http ://nces. ed.gov/forum/pub_200433 0. asp 

This publication presents a general overview of privacy laws and 
professional practices that apply to information collected for, and 
maintained in, student records. The guide provides an overview of key 
principles and concepts governing student privacy; summarizes federal 
privacy laws; identifies issues concerning the release of information 
to both parents and external organizations; and suggests good data 
management practices for schools, districts, and state education 
agencies. 



Facilities Information Management: A Guide for State and Local 
Education Agencies (NCES 2003-400) 




http://nces.ed.gov/forum/pub_2003400.asp 
This publication provides a framework for identifying a basic set 
of school facilities data elements and definitions that will meet the 
information needs of school and community decisionmakers, school 
facility managers, and the general public. It presents recommendations 
for designing and maintaining an information system that addresses the 
condition, design, use, management, and financing of elementary and 
secondary education facilities. Commonly used measures, data elements, 
and additional resources for the practitioner are also included. 



Planning Guide for Maintaining School Facilities 
(NCES 2003-347) 



PLANNING aUiDC 
'•MASNTAINbNO 
SCHOOL WiTiES 




http://nces. ed.gov/forum/pub_2 003347. asp 

This publication is intended to help school facilities managers plan 
for efficient and effective operations. It provides practical advice on 
a range of topics, including how to conduct a facilities audit, plan for 
maintenance to ensure smooth operations and avoid costly surprises, 
manage staff and contractors, and evaluate maintenance efforts. 
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